Home >> Miscellaneous
>> stream-download-batch
In this page I am going to show a very specific case study
about the way I could solve a performance bottle-neck in one
of my past academic project using Java Platform.
I had to find out the logic that is using most of the memory
allocated to the server process. This is a very simple project
with a screen to display a search screen with a date range,
start date and end date. User has to select both the start
and end date and search for all the students studied the program
within the date range. Initially number of students studied the
programme for a date range of one month was very less, but
after using this system for more than couple of years, number
of records grew exponentially and the logic used for fetching
all the records at once by a select statement resulted in
ten thousands records. And one of the functionality was to provide
user of this system to be able to download all the records in form
of a zipped file (yes a single zipped file containing all the
records). When I performed a thorough analysis on code, found
that the logic for creating a zipped file takes the complete
data in form of a List and then creates a zipped file and then
channels this complete file to the response print writer object.
|
|  |
|
So this way when complete data is passed to the web server memory,
there was potential server crash/hang was observed.
What to think? How to solve this particular issue?
These are the questions keep coming to my mind. finally after
brain storming for a day or two, I decided to do a POC on
a particular idea that came to my mind, I was not knowing
whether this will work at all or not.
This idea is not so rare, you might be thinking this while
reading this page. I decided to play around with input output
streams with batch of operation strategy.
I thought, I shall be creating a buffer output stream on top of
the ServletOutputStream object borrowed from response object,
then writing certain predefined bytes of data taken from a
batch fetch operation at data access layer. So there will be
a stream of bytes flown form the data layer to web server layer,
then this stream gets converted to a zipped output stream, and
then this zipped outputstream will be flushed to the browser with
a zipped file as the content-disposition, and content-type
appropriately defined in response header.
At first it sounds little bit confusing, isn't it!!! It did
sound bit confusing, so I started coding a POC to see
whether this is practically feasible or not within the
platform / technology stack this particular system is running
on.
I decide to write a servlet that creates a ServletOutputStream
and then sets some of those standard response setContentType
and setHeaders, then creates a BufferedOutputStream by using
the above servlet output stream as a parameter in constructor,
then there will be flushing of bytes at regular intervals.
For reducing complexities I decided to refrain from creating
the zipped output stream for some other POC :).
This following file is tested in a very limited runtime
environment, it may or may not work, and is not intended
to be some sort of advise or suggestions from my side.
it is just an example code without any intelligence, so
if use it , use it at your own risk please.
ExampleZippedDownload.java
package example;
import javax.servlet.*;
import javax.servlet.http.*;
import java.io.*;
public class ExampleZippedDownload extends HttpServlet
{
public void doGet(HttpServletRequest request,
HttpServletResponse response)
throws IOException,
ServletException {
try
{
response.setContentType("application/vnd.ns-csv");
response.setHeader("Pragma","public");
response.setHeader("Cache-Control","public");
response.setHeader("content-disposition","attachment;filename=records.csv");
ServletOutputStream sout = response.getOutputStream();
//buffer capacity is defined to be 10 as of this
//example.
BufferedOutputStream bout = new BufferedOutputStream(sout,10);
fetchRecords(bout);
System.out.println("completed downloading....");
}
catch (Exception ex)
{
ex.printStackTrace();
//Not thinking about proper exception
//handling at it is not in scope for
//this example :)
}
}
public void fetchRecords(BufferedOutputStream bout) throws Exception {
byte[] rwRecords = "Student, records".getBytes();
int i=0;
while(i<100){
//this is a place where one can pug in
//fetching records from database in
//batch operation.
bout.write(rwRecords,0,rwRecords.length);
bout.flush();
System.out.println("server processing...");
Thread.sleep(1000);
i++;
}
}
}
|
|
If anything missed out , please let me know at
techienjoy at yahoo . com
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Android ListView Example : |
Example on Android List View
explained with a very simple scenario
and article with appropriate screens
captured and shown.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Android NFC Example : |
Example using NFC using Android Platform
and source code implementing this example.
|
|
|
|
|
|
|
|
|
| Google GWT Example : |
Example using GWT and some design patterns and various
ways of implementing this example.
|
|
|
| Android Draw Example : |
Example using Draw using Android Platform
and source code implementing this example.
|
|
|
| Android Sensors Example : |
Example on Android Sensors Listed and
explained with a very simple scenario
and article with appropriate screens
captured and shown.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Android Views Example : |
Example using Views using Android Platform
and source code implementing this example.
|
|
|
|
|
|
|
|
|
| Android ListView Example : |
Example on Android ListView and
explained with a very simple scenario
and article with appropriate screens
captured and shown.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Android Gallery Example : |
Example on Android Gallery View
explained with a very simple scenario
and appropriate screens captured and shown.
|
|
|
|
|
|
|
|
|
|
|
|
| Android Examples : |
List of ANDROid examples
with source code and output
screens captured and shown.
|
|
|
|
|
|
|
|
|
|
|
| Android ListView Example : |
Example on Android ListView
explained with a very simple scenario
whereby showing folder and files with
structure and appropriate screens
captured and shown.
|
|
| Android Tab View Example : |
Example on Android Tab View
explained with a very simple scenario
and appropriate screens captured and shown.
|
|
| Android Canvas Example : |
Example using Canvas using Android Platform
and source code implementing this example.
|
|
|
|
|
| Android GridView Example : |
Example using GridView Widget using Android Platform
and source code implementing this example.
|
|
|
|
|
|
|
|
|
| Android Intent Example : |
Example using Intent from Android Platform
and source code implementing this example.
|
|
|
|
|
|
|
|
| Android Robots Example : |
Example using Borots using Android Platform
and source code implementing this example.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
