Transparent compression in RMI

If very large objects are being transferred over RMI layer, then system must do lot of work not only in serialization of the objects, but also in the transfer itself. If RMI based communication is wanted to be used across WAN environment, 100Mbit datatransfer speed is simply not available everywhere. For example, ADSL subscription lines are generally something around 2Mbit, which poses considereable limitations for the amount of data what can be transferred.

One possible solution is to wrap return values into data blocks, which transparently compress themselves when serialized over RMI.

[code lang=”java”]
/**
* Data block, which transparently handles serialization and
* on-demand compression of serialized data block
*
* @author kari
*/
public final class DataBlock
implements
Externalizable
{
private static final int MIN_BLOCK_SIZE = 1024;
private static final int BUFFER_LEN = 10000;

private static byte[] READ_BLOCK = new byte[2048];

private static DirectByteArrayOutputStream WRITE_BUFFER =
new DirectByteArrayOutputStream(BUFFER_LEN);

private static DirectByteArrayOutputStream READ_BUFFER =
new DirectByteArrayOutputStream(BUFFER_LEN);

private static final Object READ_LOCK = new Object();
private static final Object WRITE_LOCK = new Object();

private Object mObject;

public DataBlock() {
// For serialize
}

public DataBlock(Object pObject) {
mObject = pObject;
}

public Object getObject() {
return mObject;
}

public void readExternal(ObjectInput pIn)
throws IOException,
ClassNotFoundException
{
if (pIn.readBoolean()) {
// null: nothing to do
} else {
final boolean compressed = pIn.readBoolean();

synchronized (READ_LOCK) {
InputStream in = (InputStream)pIn;

if (compressed) {
in = new GZIPInputStream(in);
}

READ_BUFFER.reset();
int count = 0;
while ( (count = in.read(READ_BLOCK, 0, READ_BLOCK.length)) > 0) {
READ_BUFFER.write(READ_BLOCK, 0, count);
}

mObject = new ObjectInputStream(
new ByteArrayInputStream(
READ_BUFFER.getBuffer(),
0,
READ_BUFFER.size()))
.readObject();
}
}
}

public void writeExternal(ObjectOutput pOut)
throws IOException
{
if (mObject == null) {
// null
pOut.writeBoolean(true);
} else {
synchronized (WRITE_LOCK) {
{
WRITE_BUFFER.reset();
ObjectOutputStream out = new ObjectOutputStream(WRITE_BUFFER);
out.writeObject(mObject);
out.close();
}

final byte[] buffer = WRITE_BUFFER.getBuffer();
final int len = WRITE_BUFFER.size();
final boolean compressed = len > MIN_BLOCK_SIZE;

pOut.writeBoolean(false);
pOut.writeBoolean(compressed);

OutputStream out = (OutputStream)pOut;

if (compressed) {
GZIPOutputStream gout = new GZIPOutputStream(out);
gout.write(buffer, 0, len);
gout.finish(); // finish: this is mandatory
} else {
out.write(buffer, 0, len);
}

}
}
}

/**
* Convert RMI serialization value into DataBlock on-demand. Conversion
* is done only if conversion makes sense. Ex. primitive types and such
* are not wrapped.
*/
public static final Object create(final Object pObject) {
Object result = pObject;
if (pObject != null) {
if (pObject instanceof Number) {
// Ignore
} else if (pObject instanceof String && ((String)pObject).length() < MIN_BLOCK_SIZE) { // Ignore } else if (pObject instanceof byte[] && ((byte[])pObject).length < MIN_BLOCK_SIZE) { // Ignore } else { result = new DataBlock(pObject); } } return result; } } [/code] Notes

• ObjectOutputStream uses bitfield for storing booleans values. Thus using boolean is more efficient, than say byte values.
• Compressing very small return values doesn’t make any sense.
• Compressing whole RMI stream doesn’t work well, due to stream flushing logic.