I had a little problem with some code that went live an through a NullPointerException, causing various problems in the production system. It was far from a disaster but not great that such a thing should happen. Additionally The error was hard to track down to its source. When I did track it down I found the problem described in this post. As soon as I saw it I knew that was contributing not only to the issue but causing additional problems.
The key here was that in testing which I reviewed with the author of this code everything was OK. I would like to have unit tests for everything testing every possible issue. But we don't. Its not always practical, its not always easy to know the issues ahead of time. We should, I should insist on it. Would projects that are running late run later, most definitely. Would we have less problems once they went live, almost certainly. I just don't have the man power right now to get everything done, and remember I am not the boss. My superiors are all non technical. They want things done yesterday. They don't understand why it isn't ready just that it isn't. As long as I can rule out critical bugs it works in my favour to get things rolled out and running, and solve the minor issues as the come up. Its inefficient and its bad practice. Solution, if you are in the same position as me then get a bigger budget and hire and extra programmer or two.
With or without the extra man power there are two possible methods that can be used to avoid at least some of the issues that come up when going live with new code. Pair programming and code review. At my previous place of employment we used both to ensure as few mistakes as possible. Pair programming is an interesting method, because it requires one computer and two people. One is in the driving seat and the other sits next to them and contributes. Anyone familiar with Agile or Extreme Programming will be familiar with pair programming. When it works its great. The time lost by having to programmers work on one piece of code is gained back by having higher quality of code, that is easier to maintain, and with less bugs. Even things like the write / compile / test cycle is improved by having less syntax errors (due to them being spotted by the one not doing the driving). Ideas are generated and turned over faster. When it comes to maintenance or fixing the bugs that do get through having two people that worked on the code originally can be very helpful. If one is off on vacation or sick, or worse has left the company, the other is likely to be around. This extra ownership of the code is so helpful in these situations.
An old colleague of mine envisioned people rotating the pairs reasonably frequently, lets say one person worked with one pair in the morning and another in the afternoon. The more people you have the more possible pairs. This could mean even more that two people have some ownership of the code. Switching pairs around will give people the opportunity to take a break from each other, which is often needed in some cases, and to find really constructive and destructive pairings, to be either encouraged or discourage in the future. An additional benefit of pair programming is the lack of other distractions, you cannot be sitting in front of some code when working with someone and then suddenly check your facebook/gmail/twitter. You will certainly need more breaks, but you wont spend real coding time wasting it on the internet.
Where it breaks down is when you do not have many people it can still be done but without the variance of a large group. The number of possible pairings is n(n-1)/2 (triangular numbers), so in smaller groups this is limited. Additionally not all possible pairs work. I was often paired with weaker coders which meant I was doing most of the thinking. It was also frustrating for me not driving because the driver was working so slowly. In some projects I was working with one other programmer and we would spend some time working on the same code and other time working separately. The other coder would frequently get stuck and ask to do some work in a pair. This was code for either me working and explaining what I was doing to his code as I went along, or sitting behind him growing ever more frustrated as I told him what to do. That was a particularly nightmare case. Other pairs that I worked with were great. We thought alike, solved problems quickly together and saved time exactly in the ways mentioned above.
Code review is a lot more annoying but is a great final barrier before moving to production. I would often work with another coder on some code, then later go to someone more senior than me to be my pair for the installation. This meant that I had to explain all changes to the senior person, who would look over the code for any glaring errors, and understand the changes that I had made. Installation was made safer by having someone look over my shoulder making sure that I didn't do something stupid in production. The reason its annoying to do is again because it means taking the time of someone senior to go over something you know well again. If you want to get something in to production quickly it can be really annoying having someone asking you to justify every line of code. However in the long run this is the safest way to do things.
If you haven't tried pair programming, try it, even for a couple of hours a day.
Happy coding.
Showing posts with label programming. Show all posts
Showing posts with label programming. Show all posts
Tuesday, July 19, 2011
Friday, July 8, 2011
JVisualVM - Java's hidden monitor and profiling tool
There are a few different ways of profiling your Java application. One is using the -prof switch when calling running a Java app. This will output a profile file when the app is terminated. Another is a third party app like Yourkit. Yourkit is great, but you have to add a something to the commandline and after 30 days full free trial is its super expensive.
Finally in recent editions of the JDK there is jvisualvm.exe. Make sure you have the JDK and not just the JRE. locate the install directory and find the bin directory. There you should find the executable. Once you open it up you can go to tools and add some additional plugins. On the left hand side you will see the available JVMs that are connected. The good thing about this profiler/monitor is that you don't need to add anything to the commandline like you do with Yourkit. In your kit you have to add something to connect to that yourkit agent. Here it just connects and you can see all the JVMs running. Each instance of Java should run in a new JVM so you can run and view many different programs at once. It gives you a basic view of the memory and processor usage along details about calls to GC and the number of threads and classes being used. Additionally you can run a profiler to which will sample the application and give details of either cpu or memory usage. This is extremely useful in finding memory leaks or in my case optimizing your code. I do a large amount of number crunching which tends to use a lot of memory and cpu, using jvisualvm I have reduced the amount of memory used and made the whole process more efficient. There are other features that let you view information about classes.
I came across and a problem when running jvisualvm the other day. The executable ran and but it did not detect my running application. I ran an old version of my application and found that it was detected. In the end I have discovered that for some strange reason the java executable has to be running with code located on the same drive. Its very strange but seems to be the only solution I am aware of. If anyone else has seen this problem let me know, especially if you solved it in another way. As usual drop me line if you have questions that I haven't answered in this post.
Happy optimizing.
Finally in recent editions of the JDK there is jvisualvm.exe. Make sure you have the JDK and not just the JRE. locate the install directory and find the bin directory. There you should find the executable. Once you open it up you can go to tools and add some additional plugins. On the left hand side you will see the available JVMs that are connected. The good thing about this profiler/monitor is that you don't need to add anything to the commandline like you do with Yourkit. In your kit you have to add something to connect to that yourkit agent. Here it just connects and you can see all the JVMs running. Each instance of Java should run in a new JVM so you can run and view many different programs at once. It gives you a basic view of the memory and processor usage along details about calls to GC and the number of threads and classes being used. Additionally you can run a profiler to which will sample the application and give details of either cpu or memory usage. This is extremely useful in finding memory leaks or in my case optimizing your code. I do a large amount of number crunching which tends to use a lot of memory and cpu, using jvisualvm I have reduced the amount of memory used and made the whole process more efficient. There are other features that let you view information about classes.
I came across and a problem when running jvisualvm the other day. The executable ran and but it did not detect my running application. I ran an old version of my application and found that it was detected. In the end I have discovered that for some strange reason the java executable has to be running with code located on the same drive. Its very strange but seems to be the only solution I am aware of. If anyone else has seen this problem let me know, especially if you solved it in another way. As usual drop me line if you have questions that I haven't answered in this post.
Happy optimizing.
Wednesday, July 6, 2011
Excpetions - if you haven't caught on then you should try to
I am sure this has been said before but I came across an error that occurred yesterday. I was told the code was working. For the most part I could see that in the staging environment it was working just fine. Sunday comes around. My install day. I check the staging environment. I find that there is a null pointer exception. The code crashed at a strange point. With some deduction I realize where the error is coming from. A method was called on an object that was clearly null, hence causing the exception. The object that caused the exception was instantiated on the line before. So no doubt there. I the method called on the line before returns the object in question so I look at the method, which happened to be in a different class. I see:
public SomeObject badMethod(InputObject obj){
try {
<code some of which that could cause an exception..
including declaration of SomeObject
and return of the instantiated object>
}
catch (Exception e) {
return null;
}
}
Two big problems with this, possibly three. The minor of all of these is that so much code is in the try block. I am not sure if this is wrong, but at the same time it doesn't look right mainly from a maintenance point of view. I had to delete the try and catch lines to find out which line of code require the exception handling, thanks to Eclipse that bit was simple.
I found out that the exception was actually a ParseException.
TIP #1: When using try catch blocks to catch exceptions, catch the specific exception. Always be more explicit when you can. Code is more understandable and you have the specific exception object available to you in the catch block.
The really big problem was how this was handled. Theoretically I have no problem with returning null from a method. However if you use a method that could return null that value must be handled. Sometimes it is better if the method throws an exception instead.
TIP #2: If the method could return null you must test for it. Otherwise you will end up with a nasty NullPointerException
In my case I ended up with an unhandled NullPointerException instead of a ParseException. I had no further information about this as well. This brings me to another point, at the very least you want to have the line
e.printStackTrace();
in the catch block. At least then you can track the exception properly. Better still use a logger, and add a little message of your own perhaps with some variable values printed so that you have an idea of what went wrong in your log or on your console. With most loggers you can add the the exception itself as an argument and the logger will handle outputting the relevant information from it.
TIP #3: Use a logger to print a sensible message from the catch block.
In this example the there was no real exception handling in the code. The catch block just returns a null which is not handled in the calling code. My solution in this case was to add a sensible logging line and throw the exception again. This way the exception is passed up the stack. The key here is that it forces the calling method to handle this exception. In this case that works, I needed an extra try catch block, but the error will be handled better. Now the parse exception will be passed up through the calling stack and the method will now handle this error correctly.
TIP #4: Handle the exception, its not enough just to print the stack trace or log the error.
Conclusion:
The tips I have presented here are very important when dealing with exceptions in Java. It is too easy, especially in Eclipse which does so much for you, to leave the printStackTrace() in there and not do anything else. But they must be handled. Learn to use them problem and exceptions will be your friend. Errors will be handled correctly and the code will run more smoothly and hopefully be a little more readable.
Happy coding.
public SomeObject badMethod(InputObject obj){
try {
<code some of which that could cause an exception..
including declaration of SomeObject
and return of the instantiated object>
}
catch (Exception e) {
return null;
}
}
Two big problems with this, possibly three. The minor of all of these is that so much code is in the try block. I am not sure if this is wrong, but at the same time it doesn't look right mainly from a maintenance point of view. I had to delete the try and catch lines to find out which line of code require the exception handling, thanks to Eclipse that bit was simple.
I found out that the exception was actually a ParseException.
TIP #1: When using try catch blocks to catch exceptions, catch the specific exception. Always be more explicit when you can. Code is more understandable and you have the specific exception object available to you in the catch block.
The really big problem was how this was handled. Theoretically I have no problem with returning null from a method. However if you use a method that could return null that value must be handled. Sometimes it is better if the method throws an exception instead.
TIP #2: If the method could return null you must test for it. Otherwise you will end up with a nasty NullPointerException
In my case I ended up with an unhandled NullPointerException instead of a ParseException. I had no further information about this as well. This brings me to another point, at the very least you want to have the line
e.printStackTrace();
in the catch block. At least then you can track the exception properly. Better still use a logger, and add a little message of your own perhaps with some variable values printed so that you have an idea of what went wrong in your log or on your console. With most loggers you can add the the exception itself as an argument and the logger will handle outputting the relevant information from it.
TIP #3: Use a logger to print a sensible message from the catch block.
In this example the there was no real exception handling in the code. The catch block just returns a null which is not handled in the calling code. My solution in this case was to add a sensible logging line and throw the exception again. This way the exception is passed up the stack. The key here is that it forces the calling method to handle this exception. In this case that works, I needed an extra try catch block, but the error will be handled better. Now the parse exception will be passed up through the calling stack and the method will now handle this error correctly.
TIP #4: Handle the exception, its not enough just to print the stack trace or log the error.
Conclusion:
The tips I have presented here are very important when dealing with exceptions in Java. It is too easy, especially in Eclipse which does so much for you, to leave the printStackTrace() in there and not do anything else. But they must be handled. Learn to use them problem and exceptions will be your friend. Errors will be handled correctly and the code will run more smoothly and hopefully be a little more readable.
Happy coding.
Saturday, June 11, 2011
Programming tests: is there a shortcut
For those of you not following my tweets, I tweeted the other day about http://www.codeeval.com/. This is a great service that basically allows you to create job applications online (no big deal) with the bar to sending it in as passing the programming problem that it presents (very big deal). This could be a big help in theory. You can see how organised they are about coding, for a good enough problem it will show their problem solving skills and their thought processes.
I have a problem with this though, there seems to be no way to block the applicant looking the answers up. There are timed problems, but copy paste is pretty quick once you have found the algorithm on the web. In addition it has a plagiarism detector so you can see if candidates are sharing their answers. This could actually be used by submitting answers that you find on the web. Once a candidate uses the same answers from searching the web the plagiarism detector will flag them.
Considering all of this I would still want to do a reasonable live test as well. My main reason being efficiency of programmers is quite important to me. I cannot stand sitting with someone who is repetitively doing the same slow process over and over again. I saw recently a description of programmers as being "Inherently lazy with just the right amount of motivation." This description presents us with someone that is too lazy to keep doing something that takes 2 minutes over and over again, but will be motivated to spend an hour finding or writing a solution that will make this process automatic. I love this, its true about me, and about many other coders. I remember hearing a quote by Richard Stallman, Founder of the GNU project, who said that he would have never got it done if he wasn't lazy. I cannot find any reference for that right now so don't take me at my word. But I am pretty sure that I read it in Rebel Code. I digress...
I want to see not just efficient solutions to problems but efficient working method to get there. There is nothing more painful than watching someone click on the file menu and select copy and paste, even using right click in an editor, everyone should know ctrl^c ctrl^v. Thats ok, I doubt you would come across many programmers that don't, but there are more examples of things that people can do to work quicker. Selecting, single words and lines are very easy to select by double or triple clickling, no need to be exact about highlighting all the letters. The less a mouse is used the better. Users of VI and EMACS will be proficient at this and will probably be mouse free in other environments too. Additionally X windows users will know that if you highlight something it is copied to the clipboard and can be pasted using a single middle click. Saving too, most of the time its just ctrl^s but not doing it will mean you get prompted to save a few seconds are required of a mouse click, these things do add up. It sounds like I am talking about some pretty irrelevant or small things but trust me when you watch some one who doesn't use these things its slow.
I am not the best at these things, but I definitely notice when they are not being done. When I spent more time in EMACS and VI I taped a list of common commands on the wall behind my monitor. The more I used them the less I had to refer to the list and the faster I got at using these wonderful tools.
One weird thing that I have is about searching through code. I get it if you don't know the code, but there are many shortcuts especially in the IDEs to make it quicker. I much prefer ctrl^j to ctrl^f in Eclipse, but what I prefer the most is being familiar enough with the code to know where to go. Don't search just go straight there. Someone who can type quick enough and knows the shortcuts to search should be able to beat me in a large file. However its the familiarity with the code that I am looking for. If you are my programmer and you don't know the code well enough how can you solve bugs or make feature changes. I am not talking here about new code, I am talking about code the programmer has written themselves. Searching is fine, I would never say to someone that they cannot do it, but I want to see some indication that the code they have spent a few weeks writing is somehow bouncing around their heads.
I have a problem with this though, there seems to be no way to block the applicant looking the answers up. There are timed problems, but copy paste is pretty quick once you have found the algorithm on the web. In addition it has a plagiarism detector so you can see if candidates are sharing their answers. This could actually be used by submitting answers that you find on the web. Once a candidate uses the same answers from searching the web the plagiarism detector will flag them.
Considering all of this I would still want to do a reasonable live test as well. My main reason being efficiency of programmers is quite important to me. I cannot stand sitting with someone who is repetitively doing the same slow process over and over again. I saw recently a description of programmers as being "Inherently lazy with just the right amount of motivation." This description presents us with someone that is too lazy to keep doing something that takes 2 minutes over and over again, but will be motivated to spend an hour finding or writing a solution that will make this process automatic. I love this, its true about me, and about many other coders. I remember hearing a quote by Richard Stallman, Founder of the GNU project, who said that he would have never got it done if he wasn't lazy. I cannot find any reference for that right now so don't take me at my word. But I am pretty sure that I read it in Rebel Code. I digress...
I want to see not just efficient solutions to problems but efficient working method to get there. There is nothing more painful than watching someone click on the file menu and select copy and paste, even using right click in an editor, everyone should know ctrl^c ctrl^v. Thats ok, I doubt you would come across many programmers that don't, but there are more examples of things that people can do to work quicker. Selecting, single words and lines are very easy to select by double or triple clickling, no need to be exact about highlighting all the letters. The less a mouse is used the better. Users of VI and EMACS will be proficient at this and will probably be mouse free in other environments too. Additionally X windows users will know that if you highlight something it is copied to the clipboard and can be pasted using a single middle click. Saving too, most of the time its just ctrl^s but not doing it will mean you get prompted to save a few seconds are required of a mouse click, these things do add up. It sounds like I am talking about some pretty irrelevant or small things but trust me when you watch some one who doesn't use these things its slow.
I am not the best at these things, but I definitely notice when they are not being done. When I spent more time in EMACS and VI I taped a list of common commands on the wall behind my monitor. The more I used them the less I had to refer to the list and the faster I got at using these wonderful tools.
One weird thing that I have is about searching through code. I get it if you don't know the code, but there are many shortcuts especially in the IDEs to make it quicker. I much prefer ctrl^j to ctrl^f in Eclipse, but what I prefer the most is being familiar enough with the code to know where to go. Don't search just go straight there. Someone who can type quick enough and knows the shortcuts to search should be able to beat me in a large file. However its the familiarity with the code that I am looking for. If you are my programmer and you don't know the code well enough how can you solve bugs or make feature changes. I am not talking here about new code, I am talking about code the programmer has written themselves. Searching is fine, I would never say to someone that they cannot do it, but I want to see some indication that the code they have spent a few weeks writing is somehow bouncing around their heads.
Wednesday, April 13, 2011
The benefits of getting things right in OO
For anyone experienced in writing good OO code, the design is critical. Planning ahead is essential in creating reusable objects. I have been working on some old code that I wrote when I first started my current job and at the same time was very new to Java. Its horrible. Everything thrown in together. Control, view and models all in one class; kludges, hacks and workarounds thrown in instead of proper refactoring. Right now I am adding a new feature to the system. I had two choices glue another piece of code to the mess or refactor the whole code base while adding in this change. The result an extended project but more maintainable code, better code generally. Additionally many things that I am aware could be added or an issue have been taken care of, so that they should be easier to deal with later.
So why was it such a mess in the first place?
There are many reasons, but it comes down to perceived amount of time to finish a task. If I think I can get something done with a little hack or a quick kludge then thats the way its going to get done. I know its wrong and I know in the future it might cost me but my environment can sometimes dictate that it must be done now. I am the CTO, I am the bridge to the technical staff. My position is to get the technology working based on the requirements of my non technical bosses. The situation where my bosses think something simple in their heads should be easy and quick to implement happens time and time again
The non technical management want working systems or products. My products are two separate systems. One system is an offline system. They run it, it does something, and produces a result. The other is a live realtime system that runs 24/5 communicating with the outside world. That is what they want to see, it is my job to make sure they are produced, working and reliable. Occasionally they request changes or we have routine upgrades. This is where things get complicated. They want it done. They want it done soon. They have no understanding of unit testing, refactoring and documentation.
What is the solution?
Get it done. Many could argue getting it done means unit tests and refactoring as you go. They would say that doing testing and keep code maintained is actually efficient in the short run as well. This maybe the case but some times I am just too short sighted to see it like that. I am under pressure, I have to get it done, why would I write some more code that isnt really "relevant" right now.
I am in a good position right now. I have two working systems and though there is pressure to get this new feature into production I have more time, and I am able to fend off my bosses so long enough to refactor a large chunk of my system.
Conclusion. My code looks a lot nicer. I have implemented the MVC pattern nicely, well considerably better than before. I have reusable objects that have meant creating the two different view layers considerably easier. Small changes to the model in the future will require only making changes in one and not two spots. Its obvious now, but I am glad to see how much I have learned since I started.
So I got it done and now I have got it right. It would have been nice to get it right first time, but they next project will benefit from what I have learned. I will not necessarily get it right but it wont be as wrong as the code that I have just rewritten was.
Edit: This post was written concerning refactoring the offline system. A change was made to the live system recently that used the refactoring of the offline system. The change was smooth and easy because it simply reused previously tested and working components from the offline system. go me!
So why was it such a mess in the first place?
There are many reasons, but it comes down to perceived amount of time to finish a task. If I think I can get something done with a little hack or a quick kludge then thats the way its going to get done. I know its wrong and I know in the future it might cost me but my environment can sometimes dictate that it must be done now. I am the CTO, I am the bridge to the technical staff. My position is to get the technology working based on the requirements of my non technical bosses. The situation where my bosses think something simple in their heads should be easy and quick to implement happens time and time again
The non technical management want working systems or products. My products are two separate systems. One system is an offline system. They run it, it does something, and produces a result. The other is a live realtime system that runs 24/5 communicating with the outside world. That is what they want to see, it is my job to make sure they are produced, working and reliable. Occasionally they request changes or we have routine upgrades. This is where things get complicated. They want it done. They want it done soon. They have no understanding of unit testing, refactoring and documentation.
What is the solution?
Get it done. Many could argue getting it done means unit tests and refactoring as you go. They would say that doing testing and keep code maintained is actually efficient in the short run as well. This maybe the case but some times I am just too short sighted to see it like that. I am under pressure, I have to get it done, why would I write some more code that isnt really "relevant" right now.
I am in a good position right now. I have two working systems and though there is pressure to get this new feature into production I have more time, and I am able to fend off my bosses so long enough to refactor a large chunk of my system.
Conclusion. My code looks a lot nicer. I have implemented the MVC pattern nicely, well considerably better than before. I have reusable objects that have meant creating the two different view layers considerably easier. Small changes to the model in the future will require only making changes in one and not two spots. Its obvious now, but I am glad to see how much I have learned since I started.
So I got it done and now I have got it right. It would have been nice to get it right first time, but they next project will benefit from what I have learned. I will not necessarily get it right but it wont be as wrong as the code that I have just rewritten was.
Edit: This post was written concerning refactoring the offline system. A change was made to the live system recently that used the refactoring of the offline system. The change was smooth and easy because it simply reused previously tested and working components from the offline system. go me!
Subscribe to:
Posts (Atom)