Dear James,
I have also tried like this. Both gives me error.
private LinkedBlockingQueue<String> queue = new LinkedBlockingQueue<String>();
class ConnectionHandler implements Runnable {and
class ConnectionHandler implements Runnable {
private LinkedBlockingQueue<String> queue = new LinkedBlockingQueue<String>();
Its just an ordinary variable that you want to use from different places in your class, so put it with the other variables (Socket, DateFormats etc) at the beginning of the class definition, not inside any method..
You need to move queue stuff (queue and QueueProcessor) into your main class so you can start the queue processor just once at program startup, access the queue in ConnectionHandler, and access the queue in QueueProcessor itself. Simplest approach is just to declare the queue in the main class and make it static public (not ideal Java technique, but good enough for now).
Dear James,
I have done something like below which I dont quite get you when you say move the queue and process into the main function.
public static void main(String[] args) {
try {
class QueueProcessor implements Runnable {
@Override
public void run() {
while (true) {
try {
String data = queue.take();
if (data.equals("Please shut down now")) {
System.out.println("QueueProcessor is shutting down");
break; // exit while loop, ends run() method
}
System.out.println("Processing " + data);
// this is where you process each string of data
} catch (Exception e) {
e.printStackTrace();
}
}
}
}
final ServerSocket serverSocketConn = new ServerSocket(9000);
while (true)
{
try
{
Socket socketConn1 = serverSocketConn.accept();
new Thread(new ConnectionHandler(socketConn1)).start();
}
catch(Exception e)
{
System.out.println("MyError:Socket Accepting has been caught in main loop."+e.toString());
e.printStackTrace(System.out);
}
}
}
catch (Exception e)
{
System.out.println("MyError:Socket Conn has been caught in main loop."+e.toString());
e.printStackTrace(System.out);
//System.exit(0);
}
}
}
class ConnectionHandler implements Runnable {
private LinkedBlockingQueue<String> queue = new LinkedBlockingQueue<String>();
private Socket receivedSocketConn1;
DateFormat dateFormat = new SimpleDateFormat("dd/MM/yyyy HH:mm:ss");
DateFormat formatter = new SimpleDateFormat("EEE, dd MMM yyyy");
DateFormat inDf=new SimpleDateFormat("ddMMyyHHmmss");
DateFormat outDf=new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
ConnectionHandler(Socket receivedSocketConn1) {
this.receivedSocketConn1=receivedSocketConn1;
}No no, into your main class (sorry, don't know what you called it), not the public void main method, or inside any other method.
Dear James,
I am kind of confuse I have done some modification as below.public void run() is the one building up the single string.
class ConnectionHandler implements Runnable {
private Socket receivedSocketConn1;
private LinkedBlockingQueue<String> queue = new LinkedBlockingQueue<String>();
DateFormat dateFormat = new SimpleDateFormat("dd/MM/yyyy HH:mm:ss");
DateFormat formatter = new SimpleDateFormat("EEE, dd MMM yyyy");
DateFormat inDf=new SimpleDateFormat("ddMMyyHHmmss");
DateFormat outDf=new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
ConnectionHandler(Socket receivedSocketConn1) {
this.receivedSocketConn1=receivedSocketConn1;
}
//@Override
public void run()
{
class QueueProcessor implements Runnable {
@Override
public void run() {
while (true) {
try {
String data = queue.take();
if (data.equals("Please shut down now")) {
System.out.println("QueueProcessor is shutting down");
break; // exit while loop, ends run() method
}
System.out.println("Processing " + data);
// this is where you process each string of data
} catch (Exception e) {
e.printStackTrace();
}
}
}
}
BufferedWriter w = null;
BufferedReader r = null;
String n="";
try {
PrintStream out = System.out;
BufferedWriter fout = null;
w = new BufferedWriter(new OutputStreamWriter(receivedSocketConn1.getOutputStream()));
r = new BufferedReader(new InputStreamReader(receivedSocketConn1.getInputStream()));
int m = 0, count=0;
String line="";
while ((m=r.read()) != -1)
{
n = n + (char) m;
//n = n + (char) m;
// n = new StringBuffer().append((char)m).toString();
int i = n.indexOf("GET");
if(i != -1) {
break;
}
//System.out.println("\njoinig the N : "+n);
//System.out.println("\nM : "+m);
if (m==35)
{
w.write("$PA\r\n");
w.flush();
//processDB(n);
queue.add(n);
n="";
}
}
}
catch (IOException ex)
{
System.out.println("MyError:IOException has been caught in in the main first try");
ex.printStackTrace(System.out);
}
finally
{
try
{
if ( w != null )
{
w.close();
}
else
{
System.out.println("MyError:w is null in finally close");
}
}
catch(IOException ex){
System.out.println("MyError:IOException has been caught in w in finally close");
ex.printStackTrace(System.out);
}
}
}Dear James,
I solved it is like this. So each time one string will be send to queue processor via this method right queue.add(n);? So what do you want me to do in the queue processor programme now? Should I do the deciphering in it and save to a db ?
class ConnectionHandler implements Runnable {
private Socket receivedSocketConn1;
private LinkedBlockingQueue<String> queue = new LinkedBlockingQueue<String>();
DateFormat dateFormat = new SimpleDateFormat("dd/MM/yyyy HH:mm:ss");
DateFormat formatter = new SimpleDateFormat("EEE, dd MMM yyyy");
DateFormat inDf=new SimpleDateFormat("ddMMyyHHmmss");
DateFormat outDf=new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
ConnectionHandler(Socket receivedSocketConn1) {
this.receivedSocketConn1=receivedSocketConn1;
}
//@Override
public void run()
{
BufferedWriter w = null;
BufferedReader r = null;
String n="";
try {
PrintStream out = System.out;
BufferedWriter fout = null;
w = new BufferedWriter(new OutputStreamWriter(receivedSocketConn1.getOutputStream()));
r = new BufferedReader(new InputStreamReader(receivedSocketConn1.getInputStream()));
int m = 0, count=0;
String line="";
while ((m=r.read()) != -1)
{
n = n + (char) m;
//n = n + (char) m;
// n = new StringBuffer().append((char)m).toString();
int i = n.indexOf("GET");
if(i != -1) {
break;
}
//System.out.println("\njoinig the N : "+n);
//System.out.println("\nM : "+m);
if (m==35)
{
w.write("$PA\r\n");
w.flush();
//processDB(n);
queue.add(n);
n="";
}
}
}
catch (IOException ex)
{
System.out.println("MyError:IOException has been caught in in the main first try");
ex.printStackTrace(System.out);
}
finally
{
try
{
if ( w != null )
{
w.close();
}
else
{
System.out.println("MyError:w is null in finally close");
}
}
catch(IOException ex){
System.out.println("MyError:IOException has been caught in w in finally close");
ex.printStackTrace(System.out);
}
}
}
class QueueProcessor implements Runnable {
@Override
public void run() {
while (true) {
try {
String data = queue.take();
if (data.equals("Please shut down now")) {
System.out.println("QueueProcessor is shutting down");
break; // exit while loop, ends run() method
}
System.out.println("Processing " + data);
// this is where you process each string of data
} catch (Exception e) {
e.printStackTrace();
}
}
}
}
}Yes.
I'm going to be v busy for a bit now - I'll try to get back to you with some more architectural stuff at the weekend.
Dear James,
Ok no problem thank you. Just to explain you further. What I will do with each string I read first is decipher is get the lat and long. Then insert into db. Then check if it is in certain geo fence area(this will involve quite a number of select query and processing) then send email if indeed it is in certain designated geo fence location. There are also other time if some other alerts (based on deciphering the strings) then I will send emails too. Hope I have made it clearer.
OK, that's fine. We are heading in the right direction. Finish and test the code to extract the strings, then work on the code to do the database stuff. I'll get back to you as soon as I can.
J
OK, I'm back...
I'm worried that there are problems with where to put which pieces of code, which means the overall picture is getting lost in the detail here, so I've put together a little skeleton program that shows how the queue(s) and classes and threads can all fit together. It's really important that you get this clear in your own head before going much further.
I don't expect you to copy/paste all this, but if you study it you will see how to apply it with your own names and code.
You'll see that this uses another queue for the email. It works EXACTLY like the database one.
Why? Because if you do database work followed by email on the same thread then the database work is held up while the email is sent, and email sending holds up the database processing. By putting them in two different threads they can proceed in parallel. In the database processing, if you want to send an email, just construct the new Message, and add it to the email queue for the email processor to send.
Final comment - try to use names for variables, classes etc that explain what they are for. It makes the code so much easier to understand... eg compare
while ((m=r.read()) != -1)
n = n + (char) m;
if (m==35)with
while ((nextChar = socketInput.read()) != -1)
message += (char) nextChar;
if (nextChar == '#')...which is easier for someone else (or you, in a year's time) to understand?
Anyway, here's the template app...
import java.util.concurrent.LinkedBlockingQueue;
import javax.mail.Message;
public class GPSDemo {
// variables shared by all the classes in this application...
private LinkedBlockingQueue<String> databaseQueue = new LinkedBlockingQueue<String>();
private LinkedBlockingQueue<Message> eMailQueue = new LinkedBlockingQueue<Message>();
class ConnectionHandler implements Runnable {
// gets data from an inbound connection and queues it for databse update
public void run() { // etc
// you already have most of this in your existing code.
// it uses the shared databaseQueue variable to queue message Strings
}
}
class DatabaseProcessor implements Runnable {
// updates databaase with data queued by ConnectionHandler
public void run() {
// this is just like the QueueProcessor example I gave you
// open database connection
while (true) {
// take next String from databaseQueue and process it
// (if string == null app is closing down, so break;)
//
// if you need to send an email Message, construct it and
// add it to eMailQueue to be sent by the MailProcessor
}
// close database connection / tidy up
}
}
class MailProcessor implements Runnable {
// sends emails that were placed on eMailQueue
public void run() {
// this is also just like the QueueProcessor example I gave you
//
// do any Mail initialisation that maybe needed
while (true) {
// take next email Message from eMailQueue and send it
// (if message == null app is closing down, so break;)
}
// tidy up
}
}
public static void main(String[] args) {
// It's bad O.O. practice to put all your code in static main method.
// Create a new instance of the class, and put the code there
new GPSDemo();
}
GPSDemo() { // default constructor
new Thread(new DatabaseProcessor()).start();
new Thread(new MailProcessor()).start();
// now start the ServerSocket, wait for connections, start ConnectionHandlers
// (move your existing code to here from your static main)
//
// then, when the app is closing down and ServerSocket is closed...
// tell processors to shut down by adding a null entry to their queue
// (which they will find after processing all the outstanding items)
databaseQueue.add(null);
eMailQueue.add(null);
}
}Dear James,
I would like to add one thing extra to explain to you. My listener will keep listening and will never stop. So when must I send databaseQueue.add(null); or eMailQueue.add(null);? Currently below is how my main codes of the listener is. Should I move all my main function codes in the commServer to the GPSDemo? I feel I am wrong here. Do you think I should move new Thread(new DatabaseProcessor()).start(); and new Thread(new MailProcessor()).start(); into my connectionHandler class?
public class commServer {
public static void main(String[] args) {
try {
final ServerSocket serverSocketConn = new ServerSocket(9000);
while (true)
{
try
{
Socket socketConn1 = serverSocketConn.accept();
new Thread(new ConnectionHandler(socketConn1)).start();
}
catch(Exception e)
{
System.out.println("MyError:Socket Accepting has been caught in main loop."+e.toString());
e.printStackTrace(System.out);
}
}
}
catch (Exception e)
{
System.out.println("MyError:Socket Conn has been caught in main loop."+e.toString());
e.printStackTrace(System.out);
//System.exit(0);
}
}
}
class ConnectionHandler implements Runnable {
private Socket receivedSocketConn1;
private LinkedBlockingQueue<String> queue = new LinkedBlockingQueue<String>();
DateFormat dateFormat = new SimpleDateFormat("dd/MM/yyyy HH:mm:ss");
DateFormat formatter = new SimpleDateFormat("EEE, dd MMM yyyy");
DateFormat inDf=new SimpleDateFormat("ddMMyyHHmmss");
DateFormat outDf=new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
ConnectionHandler(Socket receivedSocketConn1) {
this.receivedSocketConn1=receivedSocketConn1;
}My listener will keep listening and will never stop. So when must I send databaseQueue.add(null); or eMailQueue.add(null);?
See demo code line 61
Should I move all my main function codes in the commServer to the GPSDemo?
Just look at the demo code - it explains what to put where.
Do you think I should move new Thread(new DatabaseProcessor()).start(); and new Thread(new MailProcessor()).start(); into my connectionHandler class?
No. That will start new processors every time there's a new connection. Put it where I showed you in the demo code. You just need to start them once.
Dear James,
I have done as below. I guess is fine as according to your idea right. I got one question here my message which is built up based on each character need more processing. I will do something like this String[] result = message.split(",");. Then based on it I will get the speed,latitude,longitude etc. Finally this will be insert and update according to the respected database fields. So where do you suggest I chunk the message in the databaseProcessor is it?
public class GPSDemo {
// variables shared by all the classes in this application...
private LinkedBlockingQueue<String> databaseQueue = new LinkedBlockingQueue<String>();
//private LinkedBlockingQueue<Message> eMailQueue = new LinkedBlockingQueue<Message>();
class ConnectionHandler implements Runnable {
private Socket receivedSocketConn1;
DateFormat dateFormat = new SimpleDateFormat("dd/MM/yyyy HH:mm:ss");
DateFormat formatter = new SimpleDateFormat("EEE, dd MMM yyyy");
DateFormat inDf=new SimpleDateFormat("ddMMyyHHmmss");
DateFormat outDf=new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
ConnectionHandler(Socket receivedSocketConn1) {
this.receivedSocketConn1=receivedSocketConn1;
}
// gets data from an inbound connection and queues it for databse update
public void run() { // etc
// you already have most of this in your existing code.
// it uses the shared databaseQueue variable to queue message Strings
BufferedWriter w = null;
BufferedReader r = null;
String message="";
try {
PrintStream out = System.out;
BufferedWriter fout = null;
w = new BufferedWriter(new OutputStreamWriter(receivedSocketConn1.getOutputStream()));
r = new BufferedReader(new InputStreamReader(receivedSocketConn1.getInputStream()));
int m = 0, count=0;
int nextChar=0;
while ((nextChar=r.read()) != -1)
{
message += (char) nextChar;
//n = n + (char) m;
// n = new StringBuffer().append((char)m).toString();
int i = message.indexOf("GET");
if(i != -1) {
break;
}
if (nextChar == '#')
{
w.write("$PA\r\n");
w.flush();
databaseQueue.add(message);
message="";
}
}
}
catch (IOException ex)
{
System.out.println("MyError:IOException has been caught in in the main first try");
ex.printStackTrace(System.out);
}
finally
{
try
{
if ( w != null )
{
w.close();
}
else
{
System.out.println("MyError:w is null in finally close");
}
}
catch(IOException ex){
System.out.println("MyError:IOException has been caught in w in finally close");
ex.printStackTrace(System.out);
}
}
}
}
class DatabaseProcessor implements Runnable {
// updates databaase with data queued by ConnectionHandler
public void run() {
// this is just like the QueueProcessor example I gave you
// open database connection
while (true) {
// take next String from databaseQueue and process it
// (if string == null app is closing down, so break;)
//
// if you need to send an email Message, construct it and
// add it to eMailQueue to be sent by the MailProcessor
}
// close database connection / tidy up
}
}
/*class MailProcessor implements Runnable {
// sends emails that were placed on eMailQueue
public void run() {
// this is also just like the QueueProcessor example I gave you
//
// do any Mail initialisation that maybe needed
while (true) {
// take next email Message from eMailQueue and send it
// (if message == null app is closing down, so break;)
}
// tidy up
}
}
*/
public static void main(String[] args) {
// It's bad O.O. practice to put all your code in static main method.
// Create a new instance of the class, and put the code there
new GPSDemo();
}
GPSDemo() { // default constructor
new Thread(new DatabaseProcessor()).start();
// new Thread(new MailProcessor()).start();
try
{
final ServerSocket serverSocketConn = new ServerSocket(9000);
while (true)
{
try
{
Socket socketConn1 = serverSocketConn.accept();
new Thread(new ConnectionHandler(socketConn1)).start();
}
catch(Exception e)
{
System.out.println("MyError:Socket Accepting has been caught in main loop."+e.toString());
e.printStackTrace(System.out);
}
}
}
catch (Exception e)
{
System.out.println("MyError:Socket Conn has been caught in main loop."+e.toString());
e.printStackTrace(System.out);
//System.exit(0);
}
databaseQueue.add(null);
// eMailQueue.add(null);
}
}Its easier to add the whole message String to the queue, then process it in the database processor. If you split it up into long/lat etc before queueing then you will have multiple values to put into the queue, which is harder than queuing a single String.
It may be a good idea to add a couple of print statements to check that the strings are being parsed and queued properly before you try to move on...
Dear James,
Here is my updates.Any comments. Am I opening the db at right place and closing it at the right place? So in this case the db connection is always open is it? I wont close is it?
class DatabaseProcessor implements Runnable {
// updates databaase with data queued by ConnectionHandler
Connection dbconn = null;
Statement stmt = null;
public void run()
{
// this is just like the QueueProcessor example I gave you
// open database connection
try
{
dbconn = DriverManager.getConnection("jdbc:mysql://localhost:3306/***?"+"user=***&password=***");
stmt = dbconn.createStatement();
while (true)
{
try
{
String message = databaseQueue.take();
if (message.equals(null)) {
System.out.println("QueueProcessor is shutting down");
break; // exit while loop, ends run() method
}
System.out.println("Processing " + message);
Date dateIn = new Date();
Date date = Calendar.getInstance().getTime();
String today = formatter.format(date);
String filename= "MyDataFile"+today+".txt";
boolean append = true;
FileWriter fw = null;
try
{
fw = new FileWriter(filename,append);
fw.write(message+" "+dateFormat.format(dateIn)+"\n");//appends the string to the file
Date dateOut = new Date();
fw.write("$PA"+" "+dateFormat.format(dateOut)+"\n");//appends the string to the file
}
catch (IOException ex)
{
//ex.printStackTrace(new PrintWriter(sWriter));
System.out.println("MyError:IOException has been caught in in the file operation"+ex.toString());
ex.printStackTrace(System.out);
}
// this is where you process each string of data
}
catch (Exception e)
{
e.printStackTrace();
}
// take next String from databaseQueue and process it
// (if string == null app is closing down, so break;)
//
// if you need to send an email Message, construct it and
// add it to eMailQueue to be sent by the MailProcessor
}
}
catch (SQLException ex)
{
System.out.println("MyError:Error SQL Exception : "+ex.toString());
ex.printStackTrace(System.out);
}
finally
{
try
{
if ( stmt != null )
{
stmt.close();
}
else
{
System.out.println("MyError:stmt is null in finally close");
}
}
catch(SQLException ex)
{
System.out.println("MyError:SQLException has been caught for stmt close");
ex.printStackTrace(System.out);
}
try
{
if ( dbconn != null )
{
dbconn.close();
}
else
{
//logger.log(Level.SEVERE, "MyError:dbconn is null in finally close", "");
System.out.println("MyError:dbconn is null in finally close");
}
}
catch(SQLException ex)
{
System.out.println("MyError:SQLException has been caught for dbconn close");
ex.printStackTrace(System.out);
}
}//finally
}//run
}Hi there
1. Yes, the idea is to create just one database connection when the program starts, and use that for all transactions, then close it when the program is shutting down. Opening a database connection is a pretty heavy process, so you don't want to do it more times than you have to.
I'm finding it difficult to follow your code when the indentation isn't right, but the idea is to create the connection, then start the while loop. For each exception you need to think about whether that is fatal or not: if its just a problem with just one record then print an error message and go on to the next record. If it's more serious then you need to give up and break out of the while loop (same as if you take a null record from the queue).
Either way, after you exit the while loop you should try to close the connection, and then let the run() method finish.
Looking quickly at your code I'm not clear as to whether you close the connection inside the while loop or after it. The best way is to break out of the while when you hit a big problem, then close the connection outside the loop. (If you close the connection inside the loop you will the try to process the next record without a database connection, which won't work!)
ps: Experienced developers do a lot of testing as they go, they don't write a lot of code then try to test it all in one go. It's much easier to test small pieces one a time.
Dear James,
So if we just open one connection it will stay for ever I am a bit not clear here. As you know my scenario I will be having thousand of data coming in at very high frequency so how will this just one db connection handle all those connection? Is it each will be separate thread? So will this one connection be able to handle the traffice? Below is my simplified version of the codes. Can you see before the while loop I put a try and open a connection inside it? Then I do a catch and finally I do a clean up job. Is this fine? Then further got a while loop and inside it is a try and catch to take the message. Then after taking I will chunk and run few sql statements. Actually how to know which sql stamement is having problem?
class DatabaseProcessor implements Runnable {
// updates databaase with data queued by ConnectionHandler
Connection dbconn = null;
Statement stmt = null;
public void run()
{
// this is just like the QueueProcessor example I gave you
// open database connection
try
{
dbconn = DriverManager.getConnection("jdbc:mysql://localhost:3306/***?"+"user=***&password=***");
stmt = dbconn.createStatement();
while (true)
{
try
{
String message = "";
message = databaseQueue.take();
if (message.equals(null)) {
System.out.println("QueueProcessor is shutting down");
break; // exit while loop, ends run() method
}
System.out.println("Processing " + message);
}
catch (Exception e)
{
e.printStackTrace();
}
}//while true
}
catch (SQLException ex)
{
System.out.println("MyError:Error SQL Exception : "+ex.toString());
ex.printStackTrace(System.out);
}
finally
{
try
{
if ( stmt != null )
{
stmt.close();
}
else
{
System.out.println("MyError:stmt is null in finally close");
}
}
catch(SQLException ex)
{
System.out.println("MyError:SQLException has been caught for stmt close");
ex.printStackTrace(System.out);
}
try
{
if ( dbconn != null )
{
dbconn.close();
}
else
{
//logger.log(Level.SEVERE, "MyError:dbconn is null in finally close", "");
System.out.println("MyError:dbconn is null in finally close");
}
}
catch(SQLException ex)
{
System.out.println("MyError:SQLException has been caught for dbconn close");
ex.printStackTrace(System.out);
}
}//finallySorry - no time to go thru your code tonight. I;ll look again tomorrow.
But yes, one connection handing transactions one at at time is a good start. Certainly opening lots of database connections all the time will only make things worse.
But... with the scalable queue architecture we are using you can start two, three, four ... database processors and they will all run in parallel taking records from the queue. Depending on your hardware that may be better than just one, but there's no code changes, so its easy to test when everything is working.
Dear James,
No problem you can go through it tomorrow. I am very curious is how this one connection going to handle thousand of request. I would like to learn the in depths of this art. So my guess is this is going to bring a significant improvement to my current application where for each new message I open and close one connection currently.
Yes, as I said before - one per connection is definitely too many, Depending on the hardware you may get the best performance with one, two .. maybe 10 connections. The great thing about the approach we are using is that you can simply start as many database processors (threads) as you like, they will all take records from the same queue and process in parallel. You will be able to benchmark the finished application with 1 .. n database processors to see which is best on your hardware.
Dear James,
I am not so clear about this statement "Depending on the hardware you may get the best performance with one, two .. maybe 10 connections. "? Another thing isnt the database thread and connection are the same what is their difference in this context?
We have the database processor class. Each instance we create opens its own database connection. We also create a new Thread for each instance, so each instance has its own connection and runs in its own thread.
In answer to your Q., a thread and a connection are not the same thing, but this code always associates one connection with one thread.
In terms of DB performance, using multiple threads for concurrent DB access can improve performance, assuming each thread has its own connection. However, with "enough" threads your code will be able to create a 100% loading on some part of the hardware (could be CPU, PATA interface, disk seeks or even memory bandwidth) and nothing will make that hardware go any faster. Adding more threads at that point just creates some extra overhead. You will have to test this on the actual hardware to find the best number of threads/connections. My guess? between 1 and 4.
Finally - I'm not the most expert person here to help with SQL problems, so let's get this thing running, then you can start a new DaniWeb topic about how the best way to construct and diagnose your SQL statements.
Dear James,
Ok I am getting better now. So actually how is this per thread per connection better off then ther older method isnt this creating more over head because more threads are created compare to the ealier method? Sorry if I am asking quite in depth as I would love to understand the underlying stuff. Did you manage to see my code in my post no. 257? How is codes I have simplified it. Thank you. So I will try to complete soon the codes and try it out.
Code in 257 looks OK so far. Have you tried to compile and run it? You should keep compiling, running, testing all the time. Do not wait to finish the code before testing, test each small part as soon as you have coded it.
Threads in Java don't create any important overhead. We are maybe talking about having up to 10 or 12 in total maybe, and that's nothing. More important - if any part of the hardware is not being used then one of those threads will be ready to use it. So overall creating a few threads will tend to maximise thoughput. Opening database connections is a big overhead, so its better to have more threads and not keep opening new connections all the time.
Dear James,
Yes I keep compiling and making sure no error then only I add more codes from my previous programme because a lot of codes with lots of sql statement. I am a bit confuse here "Opening database connections is a big overhead, so its better to have more threads and not keep opening new connections all the time." Just clear me isnt it in this method each thread is also creating a new connection wouldnt that create extra overhead? Because each instance of the database processor do create a new connection right? How different is with the other method?
Because we will create 1,2,3, or 4 threads and open 1,2,3, or 4 database connections. The original code opened one database connection for each of the thousands of inbound messages.
ps - I'm going to dinner now, so no more posts from me until tomorrow. Keep smiling!
Dear James,
So is limited to just 4 threads is it? So meaning that now it will be queue and the processing wont be so many parallel compared to earlier method? So is it that the new method will limit the thread n therefore limits on the connection? Am I right?
You cab start as many database processors as you choose, 1 or more.. The queue will work with any number of database processors. You have to experiment to find the best number.
Dear James,
I think the best let me complete the whole programme then let me test it out. Then maybe you can teach me on how to benchmark both to see how this perform better then the other.
OK. That's a good idea.
We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.