Load balancing

Load balancing takes place in every web application. Mainly by limiting the amount of parallel request. However that method fails to account for the resources(e.g. memory) used by certain requests. Even limiting the amount of certain requests is not sufficent, since two requests to different servlets can share the same resource type. Therefore a load balancing in the application itself is needed, to ensure the safe use of resources.

BLoMo allows the programmer to define how much of which resource is needed for a method call. The call is then executes as soon as the resources are available. While the method call can be queued up in a blocking manner, it is also possible to queue non blocking. That way the program can execute or queue other tasks and utilize possible wait times.


//setting up the arguments of the method call
Object[] args = new Object[1];
args[0] = configurationObject;
//storing object, method name and arguments
MethodExecutor methodExecutor = new MethodExecutor(this,"expensiveTask",args);

BLoMoChannelLoadBalancer cManager = BloMoChannelLoadBalancer.instance();

//executing the method call blocking
Object result = cManager.runExecutor("memory",500,methodExecutor);

//executing the method call non-blocking
cManager.queueExecutor("memory",500,methodExecutor);
//do something in between
result = cManager.getResult(methodExecutor);

In the example shown above the method "expensiveTask" is called. That method requires roughly 500 MB of memory. If too many user invoke that method at the same time it would crash the JVM. Therefore the load balancer "memory" is used to make sure that not too many methods are invoked at the same time. For that an administrator configures how much memory may be used at the same time by the load balancer. If that reservoir is used up the invocation is delayed until the memory was freed up. By calling runExecutor the code will pause until the method call was executed. By using queueExecutor additional tasks can be executed while waiting for the method call to finish execution.

The configuration for the load balancers is done via xml.


<loadBalancers>
	<channel>
		<id>memory</id>
		<weightLimit>2000</weightLimit>
		<staticThreads>4</staticThreads>
		<maximumThreads>20</maximumThreads>
		<timeout min="60000" max="300000" factor="600"/>
	</channel>
	...
</loadBalancers>

Here the weight limit is set to 2000(an abstract number that could be viewed as MB in case of memory). Further it is said that 4 threads are kept open to fulfill incoming calls and that no more then 20 threads run parallel(even if the weight limit is not reached yet). Defining boundarys for the amount of static and dynamic threads allows to minimize overhead due to context switches. Static threads exist at all times but they avoid the overhead of creating and destroying threads. The timeout defines a guideline on how long to wait for free resources before failing an execution. This is used to prevent deadlocks(which can be avoided through correct implementation). The timeout for an execution is calculated by multiplying it's weight with the factor and verifying the min/max boundarys.

pending tasks occupied threads current weight avg wait time avg execution time

Load balancing is mainly a work in progress and needs to be tuned to the changing needs of the application. To do that the administrator needs data from the load balancer. By adding a debug entry to the configuration of the load balancer BLoMo dumps graphs, that show those values, in regular intervals into a directory.


pending tasks	occupied threads	current weight	avg wait time	avg execution time