Software is not Jewellery

I have been testing an app for almost a month now. The app has got about 161 features and runs into about 300,000 lines of code. Plus it interacts with 2 other components written in a different languages altogether. So obviously it’s a big task testing this app. I have been running through the (incomplete) test speck, and writing results, communicating the results to the developers, reproducing a given problems so developers can resolve it. Any testing activity will involve the following major parts:

1. Identifying all usage scenarios (you could also call this writing a complete test spec)

2. Verifying that no usage path results in unstable behavior

3. If a instability is found, it should be reproduced and reported

4. Verifying that the software’s output is what the user wants

The real trouble in testing an app is the fact that all these activities have to be repeated for every build that comes around. This could be as often as twice a week. Unfortunately, it is not possible to repeat all test cases in this much time. So we cut corners here and there. The result is, we don’t have control over quality. The question arises: How can you reduce testing time? So lets go point by point here about all testing activities.

Identifying all usage scenarios

This is always THE MOST DIFFICULT TASK in testing. In a moment I will explain why. Let us say we are testing Microsoft Word. I have Office Word SP3 build ver 11.8215.8202 installed on my machine here. So here is what I did:

Steps to Reproduce:

1. Open Word. Word opens with a new document

2. Enter some random text like “The quick brown fox jumps over the lazy dog”

3. Open Find dialog (you could use Ctrl-F) -> Click over the More button at the bottom of the find dialog to open the complete functionality Find Dialog

4. Enter some text in the “Find What:” text box

Here is how it would look

5. Close the opened file through File->Close. The file closes but the search dialog is open

6. Without doing anything else clcik on Format in the bottom. User menu opens

7. Click on Font.. Option and you have your bug

You can read the message box that says that you got to save your file and there is really very less RAM on your drive. All this when you DON’T have any file open. And there is more

7. Try to click on the menu items without closing the insufficient memory dialog. Now navigate to any other application and the menu still appears as on the screen

And more..

8. Navigate back to word and try to close it by hitting the red close button. The app never closes. You can of course close it by calling task manager and killing Word.

Microsoft released three service packs for Word 2003 and I could find a bug in less than half an hour. I am not saying that Microsoft is dumb. I am just saying that there are so many many usage scenarios for Word that unknown bugs are inevitable. There is a big chance that Microsoft already knows about this bug. And they did not fix it because far too less user’s will encounter a problem like this. And fixing this bug would probably need a change in architecture so huge that it will be very very expensive. And this is reality.

If Microsoft let this bug go, even after thousands of hours of testing, there is a big chance that this could happen to many others. Any application of decent size will have bugs. Because there will be some usage scenarios that went untested and unreported before the app was delivered. Identifying all user scenarios is not only close to impossible but also impractical for the project budget. Software developers have an unspoken understating of software quality. I will try and explain some precautions we could take to make sure our software caters well to all usage scenarios.

So the question is how do we make sure we have handled all usage scenarios well. Hopefully I will elaborate on that in the coming articles.

Here are some standards that most users are accustomed to see in windows applications.

A standard menu bar. Keep the menu bar as standard as possible. Here is a standard menu item strip. Of course you will have much more in your menu bar. But here is the bare minimum.

File: New, Open, Save, Save As.., Print, Page Setup, Print Preview, Exit

Edit: Cut, Copy, Paste

Tools: Customize, Options

Help: Check for Updates, About

Display the opened file name in the title bar.

Ask for confirmation when the user deletes anything.

Ask for saving the active file if the user has unsaved changes.

A document processor (like Word or AutoCAD) should open a new file by default. A project management system (like Visual Studio) should not load any project by default.

While the user is to select an option use Radio button instead of combo boxes if there are only three or fewer options and if the number of options are not going to increase in the future. For example see the difference between the Hotmail User Sign Up form and the Yahoo! sign up form.

VS

Validate data only where the lack of validations will crash the application or make a feature unusable. For example validate all email ids in a mailing list app. Do not validate phone umbers in mailing list app.

I have been reading the book: Agile Software Development by Alistair Cockburn

It is a truly interesting book that first speaks about the role of communication in Software Development, how critical communication is for the success of any software project, how all communication is imperfect and how we still manage with it. Here is an interesting extract from the book about Software Modeling or Software Specs. refuting the principle that software development is only about “model building” Alistair says:

Some successful project teams have built more and fancier models than some unsuccessful teams. From this people draw the conclusion that more modeling is better. Some successful teams have built fewer and sloppier models than some unsuccessful teams. From this, other people draw the conclusion that less modeling is better. Neither is a valid conclusion. Modeling is a part of team communication. There can be both too much modeling and too little modeling. Scrawling on napkins is sufficient at times; much more detail is needed at other times

I feel that while making Software Specifications beautiful and accurate we fail to see what purpose of the document is in the first place. I have written quite a few Software Specifications. Every document begins with the section “Purpose” where the purpose of the document is written. Here is an extract:

“The purpose of this document is to establish clarity between the development team and the project sponsors for the project that is under consideration. The document will strive to remove all ambiguity on what the deliverables are. This document will enumerate and discuss all the various features this software will bring to its users. If there are multiple user categories, the document will discuss each of them individually. Wherever possible a pictorial representation of the user interface is made to aid the understand of the user. However, these interfaces are subject to changes if the development team finds better alternatives.

This document will also aim to give the developers access to the exact software requirements. This document does not contain any implementation details but only the functional requirements of the application. However an effort will be made to present the requirements in a format so the development team finds it easier to implement.

Further, the document aims to furnish enough detail to the application architects and leads to prepare detailed use-cases and therefore make development schedules and release plans for the entire application.”

The above extract brings three main objectives of software specs into the foreground:

1. Bring clarity between the development team and the project sponsors on the exact deliverables
2. Help the designers develop a good design serving the requirements
3. Help in making development schedules and release planing

These three, I believe are the core objectives of the requirements documentation.

User Stories are stories narrated by users about how an application or parts of application should function.

Enhancements

It is comparatively easier for users to write User Stories when they ask enhancements to an existing application. For example while asking for enhancements for Microsoft Outlook, a user may ask for more fields in the address book. A user story could be “We need a secondary email address field in the address book”. In a good user story the user would also say how this new field be used. “And the application should try to email the secondary email if the first email fails.”

New Feature

When the user needs a feature that is absent in the existing application he can point to examples in other popular applications that do something similar. For example, “We like Gmail’s Search Email function. We would like to have something similar to search projects.” And follow it with some improvements if required. Like “And we would like to search based on Date of Creation or Estimated Date of Completion”

Original Idea!

When a suitable example does not exist, it is best to write down one’s thoughts as clearly as possible and discuss it with the developers as the story is being written. This way the users and the developers are more likely to understand each other.

All softwares come with release number. Here is how I think it can be numbered. Every release will be denoted by a four digit release number of the format A.B.C.D where

A = Application Life Cycle Number
This number is a function of the number of complete Software Development Cycles the application has witnessed. One complete Life Cycle includes Development + Testing + Deployment + Usage + User Feedback + Incorporation of Feedback + Support. This number will start from zero and will move to 1 after completion of the Beta Testing stage. Every time the application complete one complete life cycle this number will be incremented. An increment in this number will reset all following numbers.

B = Feature Set Number
This number is a function of feature set of an application. It denotes the number of Major Features the application is supporting. A major feature is an independent functionality providing a unique benefit to the user. Every time a major Feature is added (or removed) to the application this number is incremented. An increment in this number will reset all following numbers but will leave the preceding numbers unchanged.

C = Features Revision Number
This number is incremented if the functionality of an existing feature is modified, enhanced or re-written after the previous major release. An increment in this number will reset all following numbers but will leave the preceding numbers unchanged.

D = Bug Fix Number
This number is incremented after every bug fix for the last major release. An increment in this number leaves all other numbers unchanged.

Any release where the first two numbers i.e. A or B change is called a major release. If the last two numbers i.e. C and D have changed it is called a minor release.

Well, we just finished work on a Project Management System for Registered Accessibility Specialists in Texas USA. It was exiting thing to work for. Just take a look at the features.

Well, I just finished work on developing a new product. It is a Windows application. And the deployment part took much more time than we had estimated. Deployment is a different kind of programming altogether. People use different kinds of technologies for deployment, but we used .NET. Here I just summarize some of the issues we ran into before we could proudly say that the product is now available for people to use.

Protection from decompilation: .NET applications decompile very easily. People mostly use .NET Reflector for that. If you send out your code without securing it, anybody can read the code. To secure an application most people use “Code Obfuscation”. Code Obfuscation changes class and variable names to trivial names. So code that looks like

    class Menu

    {

        public bool NewProject()

        {

            //Open New Project Dialog Box

            NewProjectDialog NewProjectDialogObj = new NewProjectDialog();

            NewProjectDialogObj.ShowDialog();

        }

    }

may become

    class A

    {

        public bool B() //For creating a new Project

        {

            //Open New Project Dialog Box

	    C D = new C ();

            D.ShowDialog();

        }

    }

This means that anybody who is trying to reverse engineer your code cannot understand it, but he can still use the code as it is. Another way to do it is by preventing any kind for decompiling of your .NET assemblies. You can do that by various tools. Most decompilers use the CLI header to get back the source code. Removing the CLI header makes decompiling so much more difficult. Many commercial products are available for this purpose. I haven’t used all of them as are not free! Some of them are

DotFuscator

CodeVeil

Salamander

And .NET Reactor

Limited Time Trial Version Security: In case you want distribute a limited time version of your application for evaluation; the critical task is to prevent it from running after the trail period has expired. To do this you can use the above applications. .NET Reactor has excellent support for making limited time trial versions. Or you can pretty much write your own code to do the same. You can do some of these:

Place N number of flags in the computer about

1. Installed Flag (IF) – This can be used to prevent a fresh installation your demo software if this flag is present. Hence the hacker can’t make multiple installations of the same software.
2. Install date (ID)- This will give the information of the installed date.
3. Expiry Date (ED) – This gives information of expiry date.
4. Last Access date (LAD) – The date on which the program s last accessed. If the user is trying to meddle with system date in order to run it for ever, you could probably use this flag to catch him.

Here is one of the ways you could use the flags

1. During installation set IF = true, ID = LAD = System Date, ED = ID + Trial Duration
2. During Load check
   • If Decryption of flags fails, user is hacker
   • If System Date < ED then continue else Trial Period expired, exit
   • If LAD > System date or LAD< ID or LAD>ED The person is a hacker. Else if everything is fine, LAD = System Date and continue
3. During exit LAD = System date

In reality you can use just one flag to act as ID, ED and IF. But the more flags you use it will be so much more difficult for the hacker to break into your installation. In fact you could use more than one flag for ID, ED and IF individually all having the same values but encrypted differently. You could put these values in the Registry or somewhere else in the file system. That is really left to you. Remember to encrypt thee values before you put them on the system. If the encryption is broken, you’ll probably know that you are dealing with a hacker. So you can either break your installation or do whatever you choose.

There are also other scenarios your installation will have to beat the hacker:

1. Installed files are copied to another location on the same system after expiry date.
2. Installed files are copied to a new system. Though this not a big problem since, this being trail version, you should encourage more people use it.

After all this done, please don’t forget to include a license file to put hackers under legal risk as well.

PS: The author makes no claims that the information above is foolproof. Use this information at your own risk.

Writing Software Requirements Specification (SRS) is a very very popular topic in software development. If you are already reading this, then I believe that you are not from a “Software Requirements are Useless” paradigm. You are already convinced that a SRS will greatly benefit your project. So why bother about explaining WHY to write a software documents? However like most difficult questions of SDLC, the WHY is very important. The answer for this WHY, not meant here as something to convince you write a SRS. The WHY instead will answer how YOUR SRS should look and what it should contain. What is YOUR purpose for a requirements document? All software products are unique. They cater to different domains, work on different platforms and cater to different kind of people. In some softwares are written to be very responsive and intuitive to the user’s needs (Visual Studio for example). On the other extreme are softwares that the user will never actively use it at all (Your mobile phone boot loader for example). So it will perhaps be naive to think that one SRS template will do justice to any kind of project.

What the customer needs?

Doesn’t that sounds very simple? It isn’t. There are a few reasons for this. One, the customer may himself be unaware of what he needs. Mostly customers only have a set of problems they want to resolve. Some problems are intermixed, some stand out. Some are very easy to resolve some are extremely difficult. Some are critical, some aren’t. The customer however does not know all these things. He just has a vague idea that software can fix some of his problems. To figure all these things out, most Projects have a Requirements Guy. He is the one who talks to the customer and determines what should go into the requirements document. He understands the whole objective here. If there are more customers than one and there are diverse feature requests from various people, he decides which features are more important and which are not. He plays a vital role in understanding what needs to be done. It is very necessary this guy joins the development team, once the requirements phase is over.

The SRS template is a function of the requirements and the requirements, as your team will read it, will again end up being a function of the template!

Various articles are written about this subject by

Most requirements are written in isolation from the data from the market. But that is not always a good practice. Here is the Table of Contents of a typical software product requirements document where the market data is not neglected. Accompanying is an explanation of each head.

1 Table of Contents

This contains the table of contents

2 Purpose

This section enumerates the purpose of the application. It also points out, with as much clarity as possible, the people who will benefit from this application and how they will benefit from the same. This section is a short description of the environment, the intended users etc. In this section the service provided by the application under question will be enumerated. Whether the application will face competition in the market from similar services or from any other service will also be noted here. In this context, how the competition may be over come can be briefly described.

3 Part I

This deals only with the non-technical details of the project on which the technical details will be built.

3.1 Application Overview

Describes elementary details of the application under question. For example, weather, it is a web application or a static website or a windows application etc. If required, the problem (i.e. the business opportunity) in current context should be explained. This will help the development team keep their focus, decide which features are important and which are not. Secondly, it also help as the common ground between the development team, the marketers and the customers. This section will direct all parties involved throughout the lifetime of the project. This section also contains who the users are going to be and what are the deliverables.

3.2 Business Process

This section describes how the business process works in absence of this application. It is recommended this is written in collaboration with a user who is going to use the application in future. Detail in this section will help the developers capture the requirements completely. Consider the product under question is a mail application. Gmail for example. The business process under question is about “How written communication happens?” People buy paper, buy pens, buy stamps, buy envelopes, they have a table and a chair to sit and write. And they start writing. After they are done, they get gum. They enclose the letter, stick the stamps to the envelop, walk to the post box and drop it. This is the description of the business process called Written Communication. However, a mail application may also face competition from other mode of communication as well. Telephones for example! This could be noted as well.

3.3 Software Process

This section describes how the same business process will work in presence of the application. It will describe briefly how the application integrates in the business environment enumerating the advantages it will bring about in each step. For example most mail applications perform at least these two functions: 1. serve as a mail box and 2. a contact details list. The mail box brings about fast and inexpensive written communication and the contact details facilitates a mobile and weightless address book. So decreased costs, faster communication and data mobility are the advantages of a mail application. Such advantages need to be noted here. If the system interacts with other systems that will also be noted here. It is recommended this section is further broken down into smaller parts, each part describing a small part of the application under question.

3.4 System Requirements

This section will write down the minimum things the user needs to have in order to own this application. Gmail for example needs, a computer, a live internet connection and a Google account.

4 Part II

This part contains the technical details about how the application will be divided into separate parts and how it will be implemented. It is recommended that a few mock screen shots and a few system design concepts be described in some detail. However, it is important to to note that the development team may (and perhaps will) change the design expressed in this part. The intention behind these technical description is just to aid the understanding the developers. A mail application for example, can show a few screen shots of the mail box, or the address book, or the database tables it is going to use in the same. It may also want to enumerate a few encryption technologies if it feels that they are important.

5 Appendix

If there is anything you want to refer, put here.

Firefox 2.0 introduced a new feature. A spell check INSIDE the browser. This means that when you are writing a mail, you can check your spellings inside the browser. Most browser based email clients have a spell checker. BUT, because the spell checker runs only on the server, they need to send the content of the mail to the server, check for spellings at the server end and send back the suggestions. That takes some time. So having spell check INSIDE the browser certainly makes it faster for you. Like most spell checkers the Firefox spell checker also works by underlining the misspelled words and showing suggestions once you right click on the highlighted word. Here is the screen shot of the same

They have used the context menu to show the suggestions. One problem however is that the “Add to dictionary” is too close to to the suggestions. As you can see I would like to use “with” instead of weth. However when I am hurrying through my reply I may very well click just one item below what I really need. Because, the context menu will change in length for different misspelled words your hands cannot be trained to go to the right word with practice. You’ll have to pay attention every time you do this and are bound to make the same mistake more than once. In fact I just made this mistake 3 times before I decided to write this article. Also “Add to Dictionary” is not as frequently used as Undo or Cut or Copy or any other essential word processor features. They need to come before “Add to Dictionary”. Here “Add to Dictionary” should have come after “Select All” So this design violates two principles straight away.

Number one “Make it difficult to make mistakes”. Don’t keep the “Kill Me” button very close to “Make Me Rich” button. You’ll piss people off. Number two “Give access to functionality in descending order of usage”. Keep what the user wants 90% of the time, very very accessible. The rest you can keep it else where. In fact if you have features that are only for advanced users and will be used once in 3 million years “Hide Them”.

Now coming back to the Firefox spell checker. If you actually click on the “Add to dictionary” by mistake you will want to undo it. I write “freind” 80% of the time and correct it to friend using a spell checker. If I add this word to the dictionary, next time I misspell it, I wont be able to SEE that I have misspelled a word. So, I want to “Remove from dictionary” this newly added word. Or I may be sending emails with embarrassing spelling errors! Unfortunately, THERE IS NO “Remove from dictionary”.

To remove it, I had to go to

C:\Documents and Settings\ [user name] \ ApplicationData\ Mozilla\Firefox\ Profiles\ 7akrcydl.default

and remove it manually from “persdict.dat”. I did not have to spend much time searching for the solution on the web. (Which probably means that many other people had this problem too). But I bet most users will never figure how to do that until somebody actually shows them how to do it. Accountants, Architects, Business Men don’t know (and should not know) where the Application Data folder is. In fact that is why Windows hides it by default.

So here we are at another postulate for User Interfaces. “Don’t play with user’s data”. Don’t handle user data in a way he does not understand. Always provide him functionality to edit, undo, redo or if possible even to recover his data conveniently.

UPDATE

Just discovered this. Look at how the Open Office Guys do it!

It is not so easy to add to dictionary here.