Enterprise Integration

hub

Integration Styles & Foundations

File Transfer Integration

Batch file exchange between systems via shared file system or FTP/SFTP. Asynchronous, decoupled, simple. Large data volumes efficiently transferred. One system writes file, another reads. Common formats: CSV, XML, JSON, EDI. File naming conventions with timestamps. File processing acknowledgment. Polling directory for new files. Scheduled batch processing (nightly, hourly). Simple but lacks real-time capability. Mainframe-to-cloud migrations often start here.

Similar Technologies
Real-time MessagingDatabase IntegrationAPI IntegrationStreamingEvent-Driven
Shared Database Integration

Multiple applications access common database. Tight coupling through shared schema. Immediate data availability. No network calls between apps. Schema changes affect all systems. Concurrency and locking concerns. Encapsulation violated (direct data access). Performance bottleneck on database. Legacy integration pattern. Difficult to version independently. Prefer for tightly coupled modules, not distributed systems. Often anti-pattern in microservices.

Similar Technologies
API IntegrationMessagingEvent SourcingDatabase Per ServiceCQRS
Remote Procedure Call (RPC)

Synchronous request-response communication. Calling remote method appears local. gRPC, Apache Thrift, SOAP, XML-RPC. Strong typing with protocol buffers or WSDL. HTTP/2 for gRPC performance. Blocking calls, caller waits for response. Tight temporal coupling. Network failures appear as method failures. Timeouts essential. Service meshes for observability. Client and server tightly coupled on interface. Suitable for low-latency internal services.

Similar Technologies
REST APIMessagingGraphQLEvent-DrivenAsync APIs
Messaging Integration

Asynchronous event-driven communication via message broker. Producers send, consumers receive independently. Temporal decoupling (systems don't need to be up simultaneously). Message queues (point-to-point) or topics (pub/sub). RabbitMQ, Apache Kafka, AWS SQS/SNS, Azure Service Bus. At-least-once, at-most-once, exactly-once semantics. Enables event-driven architectures. Scalable and resilient. Message ordering considerations. Eventual consistency patterns.

Similar Technologies
Synchronous RPCDatabase IntegrationFile TransferDirect API CallsWebhooks
Integration Style Comparison

Choosing integration approach based on requirements. File transfer: batch, large volumes, legacy systems. Shared database: tight coupling acceptable, single team. RPC: synchronous, low latency, request-response. Messaging: asynchronous, decoupled, event-driven. Consider: latency needs, coupling tolerance, scalability, failure handling. Hybrid approaches common (API for queries, events for updates). RESTful HTTP for external, messaging for internal. Saga pattern combines sync and async.

Similar Technologies
Single StyleAd-hoc IntegrationPoint-to-pointHub-and-SpokeMesh
Hybrid Integration Patterns

Combining synchronous and asynchronous patterns. API request triggers, async processing via queue, callback on completion. Command via API, events via messaging. CQRS: sync commands, async query model updates. Request-acknowledge-callback pattern. Webhook for async notifications. Long-running operations: accept request (202), poll status endpoint, push notification when done. Balances user experience (sync) with scalability (async). Saga pattern spans sync/async boundaries.

Similar Technologies
Pure SyncPure AsyncPolling OnlyWebhooks OnlyAll Messaging
alt_route

Message Routing Patterns

Content-Based Router

Route messages based on content inspection. Examine message body, headers, or properties. Conditional routing logic (if-then-else). Different destinations based on message type, priority, customer ID. Apache Camel Content-Based Router. AWS EventBridge rules. Spring Integration router. XPath or JsonPath for message evaluation. Flexible but requires message parsing. Dynamic routing without changing producers.

Similar Technologies
Static RoutingHeader-Based RoutingTopic-BasedDirect RoutingLoad Balancer
Message Filter

Selective message forwarding based on criteria. Discard messages not meeting conditions. Filter by message type, priority, age. Reduce downstream load. Event filtering in event-driven architectures. Kafka consumer filters. AWS EventBridge filtering. Declarative filter expressions (SQL-like). Can filter at source (publisher) or destination (subscriber). Prevents processing unwanted messages. Complementary to routing.

Similar Technologies
Process AllConsumer-Side FilteringNo FilteringTopic SubscriptionSelective Consumer
Dynamic Router

Runtime routing table updates without code changes. Control channel for routing rules. Route messages based on current system state. Adaptive routing during failures or load. Feature flags to route to different services. A/B testing with traffic splitting. Service registry integration. Database-driven routing rules. Allows operational routing changes. Circuit breaker integration. Blue-green/canary routing at message level.

Similar Technologies
Static RoutesHardcoded LogicLoad Balancer OnlyFixed TopologyConfiguration-Based
Splitter Pattern

Breaking composite messages into individual parts. Single message with multiple items split into separate messages. Batch file becomes individual records. Aggregate message into component messages. Each part processed independently. Enables parallel processing. Scatter-gather pattern uses splitter. Original message ID preserved in parts for correlation. Apache Camel Splitter. AWS Step Functions Map state. Inverse of Aggregator pattern.

Similar Technologies
Process WholeBatch ProcessingSingle MessageNo DecompositionClient-Side Split
Aggregator Pattern

Combining multiple messages into one. Collect related messages (correlation ID). Wait for all parts or timeout. Emit single combined message. Scatter-gather response aggregation. Join results from multiple services. Stateful operation (requires message store). AWS Step Functions, Apache Camel Aggregator. Completion strategies: all messages, timeout, condition. Shopping cart: aggregate items. Inverse of Splitter.

Similar Technologies
No AggregationClient-Side AggregationDatabase JoinSequential ProcessingStream Processing
Routing Slip

Message carries its own routing instructions. Itinerary attached to message. Each processor reads next step and forwards. Dynamic workflow without centralized orchestrator. Step functions as data, not code. Loan processing through multiple approval steps. Each service determines next hop. Apache Camel Routing Slip. Flexible but complex debugging. Alternative to orchestration. Choreography with explicit routing.

Similar Technologies
OrchestratorStatic RoutingService Mesh RoutingChoreographyCentral Workflow
transform

Message Transformation Patterns

Data Mapper

Field-level transformation and mapping between formats. Source schema to target schema. Handles renaming, type conversion, restructuring. Visual mapping tools (Mulesoft DataWeave, XSLT). Declarative transformations. Nested object mapping. Array transformations. Expression language for logic. Version management for mappings. Schema evolution handling. Enterprise integration tools provide mappers. Can be code or configuration.

Similar Technologies
Manual CodingNo TransformationCanonical ModelService AdapterProtocol Buffer
Canonical Data Model

Enterprise-wide common message format. All services translate to/from canonical model. N integrations instead of N². Reduces point-to-point complexity. Common vocabulary and semantics. Schema management challenge. Services maintain adapters to canonical. Versioning canonical model difficult. Can become enterprise bottleneck. XML Schema, JSON Schema, Protobuf for canonical. Alternative: context-specific models with anti-corruption layers.

Similar Technologies
Point-to-point TransformationNo StandardMultiple ModelsDomain-SpecificAPI-Specific
Message Translator

Converting between system-specific formats. Adapter pattern for messaging. Each system speaks own language, translator converts. XML to JSON, SOAP to REST, Protobuf to JSON. Protocol conversion. Legacy system integration. Translation layer between bounded contexts. Apache Camel Type Converter. AWS EventBridge transformers. Can be stateless or stateful. Often combined with routing.

Similar Technologies
Canonical ModelDirect IntegrationNo TranslationClient-Side TranslationShared Format
Envelope Wrapper

Adding metadata without modifying payload. Wrap message with routing info, security context, tracing headers. Original message preserved. Headers for routing, filtering. Correlation ID, timestamp, source. JWT token in wrapper. CloudEvents specification. Kafka message key and headers. Enables cross-cutting concerns without payload changes. Unwrap at destination. Layer 7 envelope for layer 7 payload.

Similar Technologies
Modify PayloadNo MetadataInline HeadersProtocol HeadersDatabase Fields
Content Enricher

Augmenting messages with additional data. Lookup related data from external sources. Add customer details to order message. Geocode address in shipping message. Enrich with real-time data (stock price, weather). Call external API synchronously. Query database for reference data. Cache for performance. Stateless transformation. Apache Camel Content Enricher. Keeps publishers simple. Risk: enrichment failure blocks processing.

Similar Technologies
Send EverythingClient EnrichmentNo EnrichmentJoin at ConsumerMaterialized View
Claim Check Pattern

Storing large payloads externally with reference tokens. Message contains claim check (ID or URL). Receiver retrieves payload using claim check. Reduces message size in broker. Keeps queues lightweight. Store in S3, blob storage, database. Claim check expires after TTL. Useful for images, videos, large documents. Decouples message transport from payload storage. Kafka with large messages uses this. Trade-off: extra hop to retrieve.

Similar Technologies
Inline PayloadChunkingStreamingFile TransferDirect Storage
settings_ethernet

ESB & Integration Platforms

Enterprise Service Bus (ESB)

Centralized integration infrastructure. Hub-and-spoke or bus topology. Message mediation, routing, transformation. Protocol conversion (SOAP-REST-JMS). Service orchestration for workflows. Centralized governance and monitoring. Traditional integration middleware. Mule ESB, Oracle Service Bus, IBM Integration Bus. Can become monolithic bottleneck. Contrast with decentralized microservices. Modern: lightweight integration, avoid ESB anti-patterns. Useful for legacy integration.

Similar Technologies
Point-to-PointMicroservicesService MeshServerless IntegrationiPaaS
Message Mediation

Protocol conversion, transformation, and routing in ESB. Adapters for various protocols (HTTP, JMS, FTP, SMTP). Message transformation (XML, JSON, EDI, CSV). Content-based routing logic. Header-based routing. Mediation sequences/flows. WSO2 ESB mediation. Camel routes. Centralized mediation vs adapter per service. Can add latency. Alternative: smart endpoints, dumb pipes (microservices philosophy).

Similar Technologies
Direct IntegrationService AdaptersAPI GatewayNo MediationClient Libraries
Service Orchestration

Coordinating multi-step workflows centrally. BPEL, BPMN for orchestration. Central orchestrator calls services. Knows workflow logic. Synchronous or asynchronous steps. Saga orchestration. AWS Step Functions, Azure Logic Apps, Camunda. Human tasks in workflows. Compensation for failures. Visibility into process state. Contrast with choreography (decentralized). Orchestration good for complex workflows. Trade-off: central point of failure.

Similar Technologies
ChoreographyService MeshEvent-DrivenSaga ChoreographyWorkflow Engine
Integration Platform as a Service (iPaaS)

Cloud-based integration services. MuleSoft Anypoint, Dell Boomi, Informatica Cloud, Workato, Zapier. Pre-built connectors for SaaS applications (Salesforce, Slack, etc.). Low-code/no-code integration. Hybrid integration (cloud and on-premises). API management included. Visual flow designers. Scalable cloud infrastructure. Multi-tenant. Reduced operational overhead. Modern alternative to on-premises ESB. Subscription pricing.

Similar Technologies
On-Premises ESBCustom IntegrationPoint-to-PointServerlessBuild vs Buy
API Management Layer

Gateway patterns for ESB integration. API Gateway fronts ESB services. Rate limiting, authentication, caching. Developer portal for APIs. API composition (Backend for Frontend pattern). ESB handles backend integration, gateway handles external. Apigee, Kong, AWS API Gateway + ESB. API versioning and lifecycle. Analytics and monitoring. Security policies. Modern integration: API-led connectivity. Three-tier: system, process, experience APIs.

Similar Technologies
ESB OnlyDirect ESB AccessNo API GatewayService MeshGraphQL Gateway
ESB vs Microservices Integration

Decentralized integration patterns vs centralized ESB. Microservices: smart endpoints, dumb pipes. Each service owns integration logic. Lightweight messaging (HTTP, gRPC, messaging). No central ESB bottleneck. Service mesh for cross-cutting concerns. API gateway for edge. Event-driven with Kafka. Trade-offs: ESB provides governance, visibility, but less agile. Microservices more agile but require discipline. Hybrid: use ESB for legacy, microservices for new.

Similar Technologies
Pure ESBPure MicroservicesService MeshHybridServerless
sensors

Message Endpoint Patterns

Polling Consumer

Periodic message retrieval from queue. Consumer checks for messages at intervals. Pull model. Simple implementation. Control consumption rate. Works with REST APIs (long polling). SQS polling, database polling. Configurable poll interval. Inefficient for low message rates. Delay between message arrival and processing. Good for batch processing. Spring Integration poller. Can miss messages if not careful.

Similar Technologies
Event-Driven ConsumerPush ModelStreamingWebhooksLong Polling
Event-Driven Consumer

Push-based message delivery. Broker pushes messages to consumer. No polling overhead. Immediate message processing. Callback or listener pattern. RabbitMQ push, Kafka consumer poll (but push-like). WebSocket, Server-Sent Events for browser. AWS Lambda triggers. Low latency. Requires consumer to be always available. Backpressure handling important. Reactive streams. Prefetch and buffering. Acknowledges messages after processing.

Similar Technologies
Polling ConsumerPull ModelScheduled JobsBatch ProcessingRequest-Response
Competing Consumers

Load distribution across multiple consumers. Multiple instances consume from single queue. Message delivered to one consumer only. Horizontal scaling of message processing. RabbitMQ work queues, SQS multiple consumers. Partition-based in Kafka (one consumer per partition). No message duplication (within partition). Increases throughput. Failure of one consumer doesn't stop processing. Consumer groups. Auto-scaling based on queue depth.

Similar Technologies
Single ConsumerPublish-SubscribeBroadcastRound RobinPartitioned
Message Dispatcher

Distributing work to specialized processors. Single consumer receives, dispatches to handlers. Handler per message type or command. Command pattern implementation. Spring @MessageMapping annotations. Switch statement anti-pattern vs strategy. Polymorphic dispatch. Actor model dispatching. Reduces code duplication. Central routing logic. Extensible for new message types. Alternative: separate queues per type.

Similar Technologies
Single HandlerType-Specific QueuesContent-Based RouterDirect BindingReflection-Based
Selective Consumer

Message filtering at consumer level. Consumer specifies filter criteria. JMS message selectors. Kafka consumer filters (client-side). SQL-like expressions for filtering. Subscribe to subset of messages. Reduces unnecessary message delivery. Can be inefficient (broker still sends). Better: broker-side filtering or topic design. Use when consumer needs dynamic filtering. Combine with message filter pattern.

Similar Technologies
Consume AllTopic SubscriptionBroker FilteringSeparate QueuesMessage Filter
Idempotent Receiver

Handling duplicate message delivery safely. At-least-once delivery causes duplicates. Detect and skip duplicate processing. Deduplication using message ID. Store processed IDs (database, cache). Time-window deduplication. Idempotent operations naturally safe (update operations). Non-idempotent needs explicit handling (payment processing). Kafka exactly-once with transactions. SQS message deduplication. Essential for reliable messaging. Idempotency keys in APIs.

Similar Technologies
Exactly-Once DeliveryAccept DuplicatesIgnore DuplicatesTransactional OutboxClient Dedup
verified

Reliability & Error Handling

Guaranteed Delivery

Ensuring message delivery semantics. At-least-once: message delivered one or more times (duplicates possible). At-most-once: message delivered zero or one time (loss possible). Exactly-once: delivered exactly once (hard to achieve, requires coordination). Kafka transactions for exactly-once. RabbitMQ publisher confirms and consumer acknowledgments. Message persistence to disk. Delivery guarantees vs performance trade-offs. Idempotent receivers for at-least-once.

Similar Technologies
Best EffortFire and ForgetNo GuaranteesTransactionalApplication-Level
Dead Letter Queue (DLQ)

Handling permanently failed messages. Messages that can't be processed after retries. Poison messages moved to DLQ. Prevents blocking main queue. Investigate and replay manually. SQS DLQ, RabbitMQ dead letter exchange. Alerts on DLQ depth. Separate processing for DLQ. Forensics and debugging. Expiration policies for DLQ. Prevents message loss. Alternative: dead letter topic. Monitor DLQ metrics.

Similar Technologies
Discard MessagesInfinite RetryLog OnlyAlert OnlyManual Intervention
Retry Patterns

Handling transient failures. Immediate retry, exponential backoff, jitter. Retry count limits. Transient vs permanent failures. Circuit breaker integration (stop retrying if open). Retry storm prevention. Idempotent operations required. RabbitMQ delayed retry with TTL and DLX. SQS visibility timeout. Polly library (.NET). Resilience4j (Java). Backoff strategies. Max retry attempts. Fallback after retries exhausted.

Similar Technologies
No RetryImmediate FailInfinite RetryManual RetryCircuit Breaker
Compensation Transactions

Reversing completed operations in distributed systems. Saga pattern compensating transactions. Forward recovery not possible, compensate instead. Cancel order, refund payment, release inventory. Idempotent compensations. Compensation order (reverse of execution). Orchestrated vs choreographed sagas. Workflow engines track compensation. Business process compensation. Not ACID transactions. Eventual consistency. Compensating delete (soft delete).

Similar Technologies
Two-Phase CommitIgnore FailuresManual CompensationRollbackACID Transactions
Message Expiration

Time-to-live and stale message handling. Messages with TTL expire if not processed. Prevent processing old data. RabbitMQ message TTL, SQS message retention. Use for time-sensitive data (stock quotes, notifications). Expired messages moved to DLQ or discarded. Per-message or per-queue TTL. Timestamp in message for age calculation. Consumer checks message age. Prevents resource waste on stale data. Important for real-time systems.

Similar Technologies
No ExpirationInfinite RetentionConsumer Checks OnlyManual CleanupAge-Based Routing
Store and Forward

Ensuring delivery across network failures. Sender stores message until broker acknowledges. Persistent local queue at sender. Transactional outbox pattern. Message stored in database with transaction. Background process forwards to broker. Eventual delivery guarantee. Survives sender crashes. IBM MQ assured delivery. Kafka producer acks. Trade-off: complexity and latency. Essential for critical messages (payments, orders). Alternative to synchronous delivery.

Similar Technologies
Direct SendBest EffortSynchronous OnlyNo PersistenceVolatile Queues