@jakguru/amqplib-oop-axios
amqplib-oop-axios
Welcome to amqplib-oop-axios, an innovative library designed to revolutionize how axios requests are processed in a distributed system. Built upon the robust foundation of amqplib-oop-ratelimiter, this library introduces a seamless integration between axios and AMQP brokers like RabbitMQ, facilitating a decoupled architecture where requests are sent from the requester to a remote worker for processing.
amqplib-oop-axios consists of a mechanism to generate either single-use or multi-use adapters for use with axios requests, coupled with a dedicated worker module. This powerful combination ensures that axios requests utilize the same API that you are comfortable with while benefiting from the remote processing capabilities. By leveraging an AMQP broker, amqplib-oop-axios not only optimizes request handling but also maintains the integrity and reliability of the communication process.
Example Use Cases
- Rate Limit all requests across your various services to a specific API services
- Authenticate a request to a service without using axios interceptors without exposing credentials to the requester
- Horizontally increase processing bandwidth for requests by utilizing a fleet of workers without impacting the core application
- Distribute a request workload between multiple workers while maintaining the promise chain
Important Notes
streamResponse Types are not possible since there is no direct connection between the worker(s) and the requesting adapter- The
beforeRedirectproperty cannot be set on the requesting adapter, only the worker. - The
httpAgentproperty cannot be set on the requesting adapter, only the worker. - The
httpsAgentproperty cannot be set on the requesting adapter, only the worker. - The
lookupproperty cannot be set on the requesting adapter, only the worker. - The requesting adapter and the worker maintain separate
transformRequest,transformResponse, andparamsSerializerproperties. Be careful not to run the same functionality twice or you may encounter some unexpected results. - This library uses v8.serialize and v8.deserialize for serialization of responses and event information. If you are running different versions of NodeJS, this may not behave as expected.
- Defaults set on the reqesting axios instance override defaults set on the worker
- The request / response interceptors on the reqesting axios instance and on the worker are both run in the order:
- Requesting Axios Instance Request Interceptor
- Worker Instance Request Interceptor
- Worker Instance Response Interceptor
- Requesting Axios Instance Response Interceptor
Installation
npm install @jakguru/amqplib-oop-axios
or
yarn add @jakguru/amqplib-oop-axios
Worker Usage
Import / Require the worker class
import { AmqplibAxiosWorker } from '@jakguru/amqplib-oop-axios'
or
const { AmqplibAxiosWorker } = require('@jakguru/amqplib-oop-axios')
Setting up the worker
const worker = new AmqplibAxiosWorker(
'example-queue',
{}, // amqplib connection options
{
interval: 1000, // 1 second in ms
perInterval: 10, // 10 requests per interval
autostart: true,
},
{
baseURL: 'https://some-domain.com/api',
}
)
// Optional: Update the axios defaults
worker.defaults.headers.common['Authorization'] = AUTH_TOKEN;
// Optional: Add interceptors
worker.interceptors.request.use(function (config) {
// Do something before request is sent
return config;
}, function (error) {
// Do something with request error
return Promise.reject(error);
});
Adapter Usage
Import / Require Adapter Manager
import { AmqplibAxiosAdapterManager } from '@jakguru/amqplib-oop-axios'
or
const { AmqplibAxiosAdapterManager } = require('@jakguru/amqplib-oop-axios')
Retreiving the Adapter
In order to ensure that the script doesn't hang unexpectedly, if the adapter is initialized with ConnectionConstructorOptions, the connection will be closed once the request is complete and the adapter cannot be re-used. In order to create a reusable adapter instance, you should initialize the adapter with an already initialized Connection instance.
Note: If you initialize the adapter with an already initialized Connection instance, you will need to manually close the connection when you are done with it.
Using a single-use adapter with a single-use request
axios({
method: 'post',
url: '/user/12345', // The full domain / URL is not needed because baseUrl is set on the worker
data: {
firstName: 'Fred',
lastName: 'Flintstone'
},
adapter: AmqplibAxiosAdapterManager.make('example-queue', {
// amqplib connection options
})
}).then((response) => {
// handle response normally
}).catch((error) => {
// handle errors as you see fit
});
Creating a multi-use adapter
import { Connection } from '@jakguru/amqplib-oop'
const connection = new Connection()
const adapter = AmqplibAxiosAdapterManager.make('example-queue', connection)
Using a multi-use adapter with a single-use request
axios({
method: 'post',
url: '/user/12345', // The full domain / URL is not needed because baseUrl is set on the worker
data: {
firstName: 'Fred',
lastName: 'Flintstone'
},
adapter,
}).then((response) => {
// handle response normally
}).catch((error) => {
// handle errors as you see fit
});
Using a multi-use adapter with a multi-use axios instance
const instance = axios.create({
adapter,
})
instance({
method: 'post',
url: '/user/12345', // The full domain / URL is not needed because baseUrl is set on the worker
data: {
firstName: 'Fred',
lastName: 'Flintstone'
},
}).then((response) => {
// handle response normally
}).catch((error) => {
// handle errors as you see fit
});
How it Works
The following is a flowchart of how amqplib-oop-axios handles the flow of information.
┌────────────┐
│ Queue Name │
└─────┬──────┘
│
│
│
┌───────────┐ │
┌───────► Requester │ ┌────────────────────────┐ │
│ └─────┬─────┘ │ AMQP Broker Connection │ │
│ │ └───────────┬────────────┘ │
│ │ │ │
│ ┌───────▼────────┐ ┌────────────────▼──────────────────┐ │
│ │ Axios Instance │ │ amqplib-oop-axios Adapter Manager ◄────┴
│ └───────┬────────┘ └────────────────┬──────────────────┘
│ │ │
│ │ │
│ ┌──────▼─────────┐ ┌────────────▼──────────────┐
│ │ Request Config ◄───────────────────┤ amqplib-oop-axios adapter │
│ └──────┬─────────┘ │ for specific queue │
│ │ └───────────────────────────┘
│ │
│ │ ┌───────────────┐
│ │ ┌────────────► Request Queue ├
│ │ │ └───────────────┘
│ ┌───▼─────┐ │
│ │ ├─────────────────┤ ┌───────────────────────────────┐
│ │ Request │ └─────► Request Specific Cancel Queue ├
└─────────┤ Promise ◄──────┐ └───────────────────────────────┘
│ │ │
└─────────┘ │ ┌─────────────────────────────────┐
│ ┌───────────┤ Request Specific Response Queue │
│ │ └─────────────────────────────────┘
│ │
│ │ ┌────────────────────────────────────────┐
└───┼────────┤ Request Specific Upload Progress Queue ◄
│ └────────────────────────────────────────┘
│
│ ┌──────────────────────────────────────────┐
└───────┤ Request Specific Download Progress Queue ◄
└──────────────────────────────────────────┘
┌────────────┐
│ Queue Name │
└─────┬──────┘
│
│
│
│
┌────────────────────────┐ │ ┌────────────────────────┐ ┌───────────────────┐
│ AMQP Broker Connection │ │ │ AMQP Broker Connection │ │ Rate Limit Config │
└───────────┬────────────┘ │ └───────────┬────────────┘ └─┬─────────────────┘
│ │ │ │
┌────────────────▼──────────────────┐ │ ┌────▼──────────────────▼──┐ ┌──────────────┐
│ amqplib-oop-axios Adapter Manager ◄────┴─────────────► amqplib-oop-axios Worker ◄───┤ Axios Config │
└────────────────┬──────────────────┘ └────────────┬──┬──────────┘ └──────────────┘
│ │ │
│ │ │
┌────────────▼──────────────┐ │ │
┤ amqplib-oop-axios adapter │ │ │
│ for specific queue │ │ │
└───────────────────────────┘ │ └────────────────────┐
│ │
┌───────────────┐ ┌──────────▼───────────┐ ┌───────▼────────┐
► Request Queue ├───────────────────┬─────────► Worker Job Processor │ │ Axios Instance │
└───────────────┘ │ └──────────┬───────────┘ └──▲─────────┬───┘
│ │ │ │
┌───────────────────────────────┐ │ ┌──▼──┐ │ │
► Request Specific Cancel Queue ├──────────┘ │ Job ├───────────────┘ │
└───────────────────────────────┘ └─────┘ │
│
┌─────────────────────────────────┐ │
┤ Request Specific Response Queue │◄───────────┐ │
└─────────────────────────────────┘ │ │
│ │
┌────────────────────────────────────────┐ │ │
┤ Request Specific Upload Progress Queue ◄────────┼──────────────────────────────────────────────┘
└────────────────────────────────────────┘ │
│
┌──────────────────────────────────────────┐ │
┤ Request Specific Download Progress Queue ◄───────┘
└──────────────────────────────────────────┘