Paul Michaud wrote a post here a couple of days ago on the challenge of allowing offline usage in a Saas based system. In his comprehensive discussion, he used the example of a Saas based contact management system and the complexities involved in allowing users to take their data, manipulate it offline, and then synchronise it all back up to the Cloud again.
A challenge it certainly is, even for a (dare I say it) simple system such as contact management. This reminds me of a long and convoluted discussion we had amongst our development team and clients here last year over trying to create a Cloud based ERP system which allowed offline access. Here was our ‘perfect world’ scenario:
The entire company would run their accounting system in ‘the Cloud’, with users scattered across various locations being able to use a web browser on any sort of PC or operating system to update customers, suppliers, stock and invoices. In addition, field sales agents could download a copy of the inventory database to their laptops, or even smartphones, and be able to create invoices for customers whilst sitting in their offices or warehouses. No need for expensive wireless or roaming data connections.
Sounds simple? We thought so too, initially. Until we started to analyse the workflow in more detail. Consider the following scenario:
2. Salesman A decides to download the inventory database (including current stock levels) to his laptop and go out to a customer site.
3. On customer site, Salesman A enters an invoice for 25 Widget Z’s for his customer. This invoice is allocated number 1235.
4. Meanwhile, back in the office, the rest of the sales team are still entering invoices. They entered another 5 during Salesman A’s absence, so the current invoice counter is 1239.
5. Salesman A returns to the office, only to find that when he tries to sync his data back to the main system, that he is told invoice 1235 already exists. Even worse, he finds out that one of the other invoices generated in his absence has sold the last of Widget Z’s in stock, and he will have to wait a week for the back order to arrive.
So, how do we resolve this issue with several incrementing invoice numbers? Here are some of the ideas we tossed around:
1. Have a separate incrementing range for each salesperson that went offline. For example, in the above scenario, Salesman A could have his own invoice number range that started at 2000 and kept incrementing. There will still be a collision though, when the Cloud invoice numbers reach 2000. Perhaps we could start him at 20,000? What happens if you have hundreds of offline salespeople? There will be incredible gaps in your invoice number range and I wish your debtors accounts manager the best of luck trying to reconcile that mess. Not to mention the confusion of the customer getting a statement with wildly varying invoice numbers all over the place.
2. Renumber the offline invoices as they are being synced back to the Cloud. That would be fine, however we were hoping that the sales team could print and give their customers the invoice whilst still in the field. Nothing will irritate your customer more than to get a statement with totally different invoice numbers to the physical invoices he/she has in front of them. We also had a synchronisation issue when two or more field team members were trying to sync at the same time, while someone was in the middle of entering an invoice in house. Messy.
3. Generate pseudo-unique invoice numbers each time to minimise collisions. This meant that invoice numbers were huge, GUID like monstrosities that were impossible to try and remember or communicate. We wanted to keep our invoice numbers down to 6 or so digits, which severely limited the probability of generating a unique one each time. Plus, we had the same difficulties with trying to reconcile a non-linear numbering sequence at the end of the month.
The above issues only deal with the sole problem of generating invoice numbers. We haven’t even scratched the surface of the other problem highlighted above, which was of not having real time inventory counts. There was nothing preventing a salesperson from selling items that were in stock when he left the office, but were sold out in his absence. Once again, should we:
1. Place a portion of inventory items ‘on hold’ to prevent anyone in the office selling them whilst they were ‘checked out’ by the salesperson? Not really feasible, and it would throw your JIT inventory management out of kilter severely – even if we weighted the hold quantities depending on what that salesperson’s usual customers bought.
2. Deliberately lower the inventory count on the salesman’s offline database to prevent ‘overselling’. This also defeated the goal of most of the salespeople trying to move ‘end of life’ products or deliberately clearing out stock.
So you can see here, that when you start to look at real world applications in the Cloud, the issue of offline access is certainly not a simple one. It requires deliberate planning, careful execution, and sometimes just the ability to say “No, we can’t do that” and move on, much to our chagrin.
I am incredibly lucky to have smart colleagues here at CloudAve, as well as smart readers who ‘get’ Cloud Computing. Do any of you have an inspirational solution that we have missed when it comes to the above issues? If so, I would love to hear about it in the comments.
(Image credit: NathanaelB via Flickr)
(Guest post by Devan Sabaratnam)
Devan,
You are absolutely right that offline access brings a lot of challenges and often is saying No, we can’t do that the best solution. It provides stability and accuracy what is extremely important for a efficient business process.
The best alternatives are probably hybrid solutions where the offline environment can still connect on demand to the cloud when necessary. In this case the data transfer is limited and other technologies can be used.
If you take your example of the invoice number the offline client could just request a new number in sequence from the cloud. This request could use different technologies, for instance also text messages on a phone… Alternatively you could allocate a batch of numbers for the local environment and refresh the batch on demand. Say with 10-100 at a time.. depending on the average volume of invoices a day per sales person.
Suc6 with the solution…
Hi Devan,
Actually for the invoice issue you mention, the best way is not to use an auto incrementing invoice number. There are a couple of approaches that work. Mainly, you need to use a unique key that can be generated independently by each instance and still be guaranteed to be unique. I have seen two main approaches:
1) Use a GUID. its guaranteed to be unique and won’t clash. Downside. ugly to read so often map it to a human readable invoice number using some schema later.
2) Use a partitioned key gen of some kind. for example each application instance has a uniqueID or user the userID in the instance. Append to the instanceID an invoiceID. For example: InstanceID-InvocieID. Again should be unique always and more readable.
Just remember don’t use the database autoincrementing key for any distributed system or one that needs to scale huge. it will be guarenteed to be a problem if you do. (I just know I am going to get flamed by a bunch of DBA’s for that now
)
For very high volume systems we tend to go with solution 1. Remember if you only allow 6 digits then you can’t sell 1 million items. Will cause problems later.
For the inventory update issue, short of forcing a synch before doing a sale, there really is no way to keep the records coherent that I can think of.
Paul
Paul
Thank you Aad and Paul for your input. Some great ideas here.
For the record, I agree with you Paul, about the auto incrementing keys for the database – we used to use them religiously but are moving away from them to GUID style keys because, as you pointed out, they just don’t scale when you start looking at load balanced server and replication etc.
Both your ideas about partitioned keys, and requesting the next invoice no. via SMS etc. are pretty cool, and I will get out development team to investigate those.
Cheers, Devan.
I developed a solution for a customer with high inventory rotation and provided them with USB Data Cards which are not that expensive.
So most of the time they are online.
It is based on Adobe Air which has a local DB. So when it is off-line it uses the saved inventory and saves the invoices ( orders ) in the local computer.
In this case invoices need managment approval, so we only provide a receipt, and don;t have to worry about invoice numbers. But I guess separate numeration is a good option.
When the computer goes online, then inventories are automatically sync-ed and invoices are sent for approval.
Once they are approved they are sent to the customer.
In inventories we use a couple of fields to control problems, first a max sale amount that can be adjusted per product once inventory runs low. This will control the over selling of an item.
The other field we use is one that requires salesman to call the office when trying to add it to an invoice.
Adobe Air is a great tecnology that allows for complicated interaction to be presented to the user in a very simple manner.
On the last part, there will always be human interaction and process. Where sometimes problems need to be fixed by humans with a commercial edge.