Category Archives: Information Technology

My work

by

User experience for business employees

Business employees

Who am I referring to exactly?   After all even the IT developer who builds and maintains the enterprise system is an employee of the business.  In fact, I mean folks that are not IT employees.   I am referring to people whose primary knowledge is the business of the enterprise, and not computer related skills.

For instance, she is a certified underwriter of farm policies.  She is not expected to know SQL.  She is not expected to be able to setup CRON jobs, or put together a Lucene query, or even an advanced Google query.  She does not tune the Oracle database where her policies reside.

What does it mean for such employees to have good user experience?

 

User experience

Common characteristics

At the outset, the common characteristics, as described in these posts, apply to business folks too.

Besides these, there is a perspective that applies to business folks specifically, I believe.

No training

Business resources must really only need training, experience, and expertise in the business, and not in whatever enterprise-wide computer system that is in place.

For instance, an underwriter should be able to come into a new insurance company, and knowing no more than how to use a keyboard, mouse, and perhaps a touch screen interface, and without formal training, should be able to very quickly learn and be productive using the existing enterprise system.

Anything the business resource has to learn, she must be able to learn painlessly, by just using the system.

The interfaces that the business user encounters must be a clear, natural, and seamless representation of her knowledge of the business.   The system should guide the user down paths that are instantly familiar, and obviously correct.

The enterprise system must leave little room for mistakes.   Even when the mistakes happen, they must be caught early, and there must be little or no cost.  This is essential to facilitate experimentation, and self-learning.

Transparent

You know that the user experience is good when the enterprise system recedes into the background.

The computer system must not register in the user’s mind as an obstacle, as a challenge, or as anything at all that is above and beyond her knowledge of the business.

Granted, regardless of how convoluted a system is, once a user learns it, the system will recede into the background.  That is almost unfortunate, because that is how a lot of clunky systems come to be.

However, say you have to make a change to the system.  Or say you want to replace the system.   How much resistance do you encounter?  If the business complains about having to learn a whole new system all over again, your user experience is suspect.

To put it another way, your system’s interfaces must not add any cognitive burden, beyond that which the business expertise itself requires.

How to get there

The fundamental business

Good solution design begins with effective business analysis.  Before diving into solutions, business analysis must first describe the essential business problem.  Volere, a Requirements and Business Analysis consultancy has a good definition of such analysis, which it calls ‘systemic thinking’.  To paraphrase Volere, you want an understanding of the essence of the business, without being prejudiced by any solutions, whether digital, or the old-fashioned kinds.

In a lot of my past work, the product of business analysis included 
someone's notion of a solution too, typically a user interface 
designed by folks with knowledge of the business, and good intentions, 
but not much else.  

Folks with the expertise (in user experience design) to translate the 
essential business rules, processes, and outcomes, into a transparent 
computer solution, never got a chance to understand the business at all.  
Result, more often than not, avoidable errors, unnecessary iterations, 
and ultimately, an interface, that was not useless, but that was less 
optimal than it had to be.

These were common refrains - "They will get used to it", "This is a 
documentation issue", "This is a training issue", etc.  All telltale signs 
of user experience that has room for improvement.

Human interaction design and construction

There are folks with the expertise to take the description of the essence of the business that systemic business analysis produces, and design a system with the characteristics described above.

Much of this expertise has been codified, as guidelines, patterns, and frameworks, which a competent generalist can learn as necessary.

Here is a list of resources that must serve as our guides.

Further, design is an iterative process, which will include the following sort of cycle.

  • The human interaction designer comes up with a design.
  • The design is implemented as some kind of prototype.
  • Users use and evaluate the prototype.
  • Tweak, enhance, start over, until everyone arrives at a satisfactory destination.

As this suggests, besides the design expertise, you have to be able to repeatedly construct, deploy, review, and change these solutions quickly.

Construction skills include the following.

  • Create plain wireframes with a tool like Balsamic.
  • Create colors and images laden, HTML, and CSS mockups with tools like Dreamweaver, and Photoshop, etc.
  • Create live prototypes with RAD frameworks like Ruby on Rails, or Django (Python), or Grails, or Play with Scala etc.  In particular, my personal interest is in the Java eco-system (Grails, Play), and a pure Javascript solution (for instance, Bootstrap.js, and BackBone.js at the client, and node.js in the backend).

You have to have the infrastructure and the skills for continuous integration, and continuous release.

Part of the review capabilities must include the ability to run usability tests.

Finally, there will be times when interfaces will have to change deep into the construction of the system.  Your engineering must be such that the interface can change quickly, without adversely affecting the backend.  You never want to say to the client – “It is too late to make that UI change.  You should told us this earlier,”.

by

As a generalist, what must I know regarding ‘transactions’

I believe, a generalist, or a team of generalists, must offer these skills, related to the implementation of ‘business transactions‘.

The failure of a business transaction must still leave the system in a safe, and valid state. As a ‘generalist’ I must either know how to achieve that goal, or know where to quickly find a solution.

Platforms

As always, I want to be able to solve this problem in two platforms – the Java eco-ssytem, and Node.js.

 

Short-lived business transactions

Most of us are familiar with ACID transaction support in relational databases.  Relying on this support is only recommended for short-lived business transactions.

Database transactions are typically implemented by locking database tables for a particular user, which forces all others to wait till the locks are released. This necessarily slows the system down, among other complexities. Hence the recommendation that ACID transactions be very short-lived.

Here are some examples of short-lived business transactions.

  • Change the address of a customer. Typically you have already gathered the new address, and you simply have to update a few tables with the new data.
  • Apply a payment against a policy. Again, the whole transaction is typically an update to a few tables.

In the Java eco-system, this sort of short-lived transaction, when applied against a single database, is implemented with the JDBC API. We must be able to able that, using Java, Scala, and Groovy.

I must be able to talk to relational databases, and manage ACID transactions against a single database, using node.js.

 

Long-lived business transactions

However, often, there are business transactions that are long-lived, and must behave gracefully.

Here is one example of a long-lived transaction – migrating old insurance policies from one system to another.

This is historical data, often several years worth and can be voluminous. It contains many different parts, like contacts, coverages, changes made to the policy over the years, documents, etc. Accepting all of the policy into a new system can take a significant time. We used to run into policies that took 20 seconds to process completely. Things will go wrong, and when they do, you would like to cleanly rollback all of the incoming data. However, you cannot keep an ACID database transaction open for 20 second, 10, or even 5 seconds. That locks up database tables, which in turn will severely diminish your ability to handle load.

Here is another example – business workflows that extend over several days.

They are initiated, passed around to several folks, and then eventually completed. If for some reason this business process ends in some kind of rejection, or failure, you may want to discard, or perhaps archive data that this workflow created.

Consider that new accident information is received for an automobile policy. Maybe some documents are uploaded. Some premium adjustments are made. Some bills are generated. Underwriting, and billing managers review, and sign off. This process may take several days to finish. Say after doing a lot of work, you discover all this work was done on the wrong policy – perhaps an ex-husband’s. How do you roll back this work, and data that has been accumulating over several days? Surely not with a database transaction.

So how do ensure that such operations are well behaved? If necessary, how to ensure that these long-lived business transactions exhibit ACID properties?

As a ‘generalist’ I must know standard approaches, and solutions to this problem. Further, I must be able to implement these solutions in the Java eco-system, and in node.js.

 

Single database vs. multiple

In a large, heterogenous environment, you are often working with several databases. Perhaps all documents are in some legacy SQL Server DB, and day to day transactional data are in a fast MySQL DB.

How do you implement ACID when a business transaction, even a short-lived one, works with data that is distributed across more than one database?

In the Java world, there is the JTA API, which supports so-called ‘distributed transactions’. But very few people seem to use this. As a ‘generalist’ I must know what the alternative is.

Similarly, I must know how this problem can be handled in node.js.

 

Polyglot persistence

This is just a special case of the multiple database scenario. The data repository can be anything at all – relational DB, NoSQL DB, text based index, messaging end point, flat disk file, etc.

How do you implement ACID when the business transaction works with many different kinds of data repositories?

For instance, say you are recording an auto accident. Photos of the car might go into a noSQL database, like MongoDB. A description of the accident is saved to Oracle, and this information is also parsed and pushed into Lucene. Finally, a notification is dropped into a queue that a claims adjuster is watching. If this transaction dies for some reason, you want to rollback the changes you made to each of these very disparate data repositories.

by

As a generalist, what must I know regarding ‘performance’

I believe, a generalist, or a team of generalists, must offer these skills, related to the performance of software systems.

We are either going to be working with new software, which we write, or we will be working on a preexisting, complex and heterogeneous system.

A new system

On a new system, I am going to need the following skills.

  • Create a flexible profiling infrastructure.
    • Be able to log the turnaround time on relevant functionality, and at relevant layers of the system.
    • Be able to turn these logs on and off, increase and decrease the granularity of these logs, etc.
    • Create log output that is easy to analyze.  Bottlenecks must be easy to find.
  • Architect the system such that, bottlenecks can be replaced, without affecting any other part of the system.
  • Load test the system.  Throw large amounts data, and users at the system, in order to expose its fault-lines.

An existing system

More often than not, you will have to deal with preexisting systems which you cannot instrument.   Hence performance measurement will essentially have to be a black-box affair.  In such situations, these skills are needed.

  • Load test the system from whatever interface is exposed to you.
    • Throw users, and data at the available GUI.
    • Throw users and data at available APIs.
  • Specialized knowledge on profiling the various parts of the existing system will help.  For instance, say you have a PHP app running in Apache.   Knowing how to manage Apache profiling, will help.   But you cannot anticipate what tools a client will use.  So this expertise is something we have to be able to acquire quickly, as needed, and on the job.

What platforms and tools exactly?

My interest is in two platforms – the Java eco-system, and node.js.  Much of my programming experience has been in the Java world, and I am newbie to node.js.    In any event, the above requirements translate into the following concrete skills, I believe.

  • Implement profiling (logging by another name) which can be configured at run-time.
    • In a system that runs on the JVM, which includes these languages – Java, Scala, Groovy
      • Through common logging frameworks – log4j, JDK logging, SL4J
      • Through aspects – AspectJ
      • Through meta-programming – Groovy, Scala
    • In node.js
  • Break the system into modules whose boundaries are milestones that are relevant to performance.  Implement these modules such that we can look into and change the performance on any one of them, without affecting anything else.
    • For instance, presentation, service, data access layers.
    • Or web service access to an external site, a computation performed in a rule engine, a search query against a text-based index, a MapRequce query against a NoSQL db, etc.
  • Get turnaround times out of at least commonly used tools, which are listed below.  Clients may use other tools.  So this is often going to be something you figure out on the job.  Also, getting profiling information is only one problem.  Actually administering these tools to improve performance is often a complex, and specialized job.   It is almost impossible for a generalist to master all these tools.  At most, when the need arises, we need to know how and where to look for a solution.
    • Web / App Servers
      • Apache HTTP server
      • Jboss Application server
    • At the client
      • Rendering engines, and Javascript engines on web browsers (Chrome, Safari, Firefox, IE, Opera)
      • A mobile app – Android, and iOS
    • Data repositories
      • Oracle
      • MySQL
      • Lucene
    • Others
      • JBPM (Business process management)
      • Jboss Rules (Rule engine)
      • CXF, Axis (Web service frameworks)
  • Load (data and users) test.  There are plain old test automation tools, and there are specialized load and performance testing tools.  I, in fact, do not know what is adequate for this purpose exactly.   I have used simple Junit based tools in the past for this.
    • By driving a web UI that is in some browser.
    • Through a mobile app – Android, and iOS
    • By driving an API (JVM language, or node.js) directly, without going through a UI.

     

Now what?

On second glace, this is no trivial list of skills.  If nothing else, there should be little room for boredom.

by

User experience in an enterprise system

I have a decided fixation on ‘user experience‘.  However, I couldn’t tell you what I am looking for exactly.  This is an attempt to remove the fuzziness.

It seems to me that there are two primary questions.

  • Who are the users anyway, of an enterprise system?
  • What should these users be able to expect from a system?

The users of an enterprise system

Anyone who interacts with the enterprise system in any way at all, I consider its users.

  • Employees who use the system to conduct the business of the enterprise.
  • Tech folks who create, maintain, enhance, and run the system.
    • Managers
    • Programmers
    • Testers
    • Dev Ops (Software configuration folks, System Admins)
    • Software Operators
  • Business partners who use the system
  • Customers who use the system

All of these folks, in their various capacities, must have good ‘user experience‘.   What does ‘user experience‘ mean to each of these folks?

Common to all users

In this post I will only list behavior that I believe applies across all classes of users.  In subsequent posts, I’ll go more into what might make sense for various individual classes of users.

Business functions must exhibit the following characteristics.

  • They must be performant – must execute just as fast as necessary, regardless of the load on the system.
  • Failure of a function must not leave the system in an incorrect state, which might require invasive, and manual cleanup.  Rollback as necessary.
I've been involved in exercises where dealing with the errors that 
production code left in its wake, became a full time job.  There are 
many reasons why a development task ends up here, including holes in 
business analysis, code construction, and testing.  None of this is 
rocket science.  The relevant skills can be acquired, and these potholes
can, and must be avoided.
  • Every business command must be available through three avenues – a GUI, a DSL that you use at the command line, and as pluggable units in external systems like schedulers, ESBs, and workflow management systems.
A single API must support all these various methods of invoking business
functionality.

You are taught to create interfaces that are appropriate to all levels 
of expertise.  Stepping through 8 web pages in order to make one edit 
on the 9th page might be acceptable for non-tech savvy customer service 
rep.  However, a developer, who is dealing with a customer support issue, 
does not have to deal with the overhead of all that UI.  Two lines of 
script ought to do the same job just nicely.  You could use SQL and change 
data directly in the system's database.  But wouldn't you rather go 
through the application's infrastructure, which presumably has essential 
checks and balances in place. That is where the domain specific language 
(DSL) comes in.

A DSL would also enable functional testing, while avoiding the drudgery of 
dealing with the UI.
  • You must be able to access the business function from the device of your choice – desktops, laptops, tablets, phones.
This is where the world seems to be heading.  Either get on the bandwagon, 
or be left behind.
  • Command line access must be in the form of a domain specific language (DSL) with which you can write scripts to tie together various functions in a workflow.
  • Long lived commands must have these behavior.
    • Run them in the background, and bring them into the foreground as necessary.
    • Expose current status, and progress.
    • Allow you to abort in the middle of execution, and return the system safely to the previous state.
    • Allow you to stop and resume where you left off.

Required skills 

Not every project, at every client, will ask for all the features described above.  However, I believe it represents a set of skills that an engineering outfit can reasonably be expected to have.  I want to be able to deliver this functionality.

There is little here that is exotic.   I only see three or four main streams of skills, all of which are well understood, and well supported.

  • Learn to measure, manage, and deliver performance.
  • Learn to architect business functions such that they lend themselves to manipulation from the command line, and GUI.  This mostly consists of creating well-designed APIs.
  • Learn to create GUI in a browser neutral fashion, and for various mobile devices.
  • Learn to create well-engineered and expressive DSLs.

Is it practical for a small and competent team to be able to develop these competencies, and employ them  successfully?   I would have to think so.

 

by

After trying to conserve bandwidth for a few days ….

I looked around for help on slowing down the speed at which I was eating up my monthly allotment of bandwidth, and found that many other people have tread this path before. Here is one pretty nice reference at Million Clues.

As various references suggested, I switched to FireFox, and installed two Add-ons, AdBlock Plus, and ImgLikeOpera.

I don’t have any scientific proof that this has reduced my bandwidth consumption, but I can certainly see that both banner ads (AdBlock Plus), and images (ImgLikeOpera) are not fetched.   Pages are loading faster.

ImgLikeOpera has the more significant impact on my user experience.  I have set it to block all images by default.   It has been interesting going about my business on the web without any images.   Some types of sites accommodate the missing images much better than others.

EspnCricinfo looks bad without images.  They use images for backgrounds of menus, without the background image, the menu items simply get mixed up with the other text.

Shopping sites were hard to deal with, which I suppose is natural.  After about 10 minutes on the Croma site, I gave up and turned on all images.   Large shopping sites like Zappos, and Amazon, will not work very well on low bandwidth.   On the other hand, I imagine that their mobile sites are designed with bandwidth constraints in mind.

News sites did much better.   The New York Times site was lovely.  Without images it continued to look very familiar.   Missing images were clearly marked, and stayed out of the way of the text.  The text itself was in exactly the same places that it always is.  Washington Post was a little uneven.  Talking Points Memo was clean.   Political Wire never has news images, and isn’t that a nice decision.

I have a renewed appreciation for the tech and design folks at the New York Times.   They probably wouldn’t even hire me as a janitor.

Facebook was interesting.   At first, it was very hard to use with all images turned off.   After a couple of days, it is beginning to feel natural.  I guess the brain adjusts.  It gets used to the visual cues that the images provide.   After a couple of days without images, you begin to learn the cues that the other content provides.

I have deliberately stayed away from many of the Indian sites.   My experience these past few weeks at these sites has not been very pleasant.   Their attention to user experience, to put it mildly, is uneven.   Dedicated E-commerce sites (FlipKart, InfiBeam, SnapDeal, etc.) have been generally much better than enterprise sites (Tamil Nadu Electricity Board, Tata Docomo, Karur Vysya Bank, etc.).

This is all very interesting.  I have never had to take available band-width into account when coming up the design for a web UI.   I wonder how seriously this is addressed by developers who cater to folks in the first world.   However, everybody is moving towards ‘mobile design first’.   One of the basic constraints of that world, is limited bandwidth.  Decisions made for the mobile design will bleed into the desktop, and that will address bandwidth shortage in a natural manner, I expect.

 

 

by

Dealing with low Internet bandwidth, in India

In India, Internet bandwidth is limited, and the more you want, the more expensive it is.  We know that carriers in the U.S. are going in this direction too.

I got myself what is known as a “data card” (essentially a barebones phone, which plugs into your USB port, and allows you to dial out to the internet) with a pretty high band-width plan.  The promise is 3 GB per month, at more than decent speeds, and after I go past the 3 GB limit, the speed is throttled down significantly.

3 GB per month seemed adequate when I signed up.   In the U.S my mobile plan only has a 2 GB limit, and I barely use half of it each month.   I had that in mind when I took the 3 GB plan.   However, in the two weeks since I got the data card, I have used up close to 2 GB already.  I did not stream any video, or audio.  All I have been doing is reading news, and trying to get some work done online.  Needless to say I was shocked.   It appears that I forgot that most of my usage in the U.S happens over my landline, and through my work.   Now, I am forced to pay attention to how I use the Net.

This morning, I was paying some bills through my bank’s online bill payment service.  For the first time I noticed how I heedlessly waste bandwidth.   I would enter the amount for one bill, submit that, which took me through a couple of pages, and then start my next bill, which would take me through those pages all over again.  In fact, my bank allows me to enter payments for several bills at the same time, which would take me through those same two or three confirmation pages, but I would be visiting these pages only once.

I was reminded of how my mother gathers several errands together before she ventures into the city.  Any trip from our home in the southern outskirts of Chennai, a place called Kelambakkam, into the city and back, burns through a whole day.  It simply is not practical to expect to travel into Chennai two or three times a week.  Nothing else would get done at home.   You have to try and get a lot of work done on any one trip.

Another thought occurred to me while I was at my bank’s web site.  All I really want to do is specify a biller, a payment amount, perhaps a payment date, and say, go.   If the bank had a RESTful API, which it exposed to me, I could get this done in 3 lines of script.  Pick your poison – Javascript, Python, Groovy, PHP, whatever.  Imagine how little bandwidth that would require.

However, this will work only for the geeks.   Regular folks need a UI.

Ah, and suddenly, mobile apps make sense.   With mobile apps, presentation happens completely at the client.   None of the artifacts (images, bloated Javascript, CSS, etc.) that make up the presentation will travel between the client, and the server.   Rather, only the data, and a command or two will, just like with an adhoc script I might cook up to talk to a RESTful API.

I should find out exactly what is eating up the bandwidth.  I wonder how much the damned advertisements consume.

by

Look to the culture, Luke

Five of us went to Tirumala last week. We walked up Srivari Mettu, which is a flight of steps (2400, I think they said) that goes straight up the back of the mountain on top of which Tirumala sits. The climb was challenging, but not impossible.

At Tirumala, accommodations had been arranged in lodgings that were represented to us as “very good”. We got to the rooms, and to little surprise, found them, not so good.

The room, an apartment really, was large. It had three bedrooms, one of which had a window air-conditioner, two bathrooms, a dining room, and what appeared to be space for cooking. The rooms were fairly clean, and the sheets looked washed.  That is the good news.

The paint had peeled off in several places, and these spots were patched in different shades of the original color. There were several large stains on the walls. One bathroom’s geyser did not turn on, and the other bathroom did not have a knob on the hot water tap. The doors to both bathrooms were pockmarked with rust, and the bottom of the doors were actually eaten away.

The rooms looked, and felt, sadly, rather shabby.

The room cost 2000 rupees I think, which ain’t exactly peanuts.

Tirumula is reputedly one of the richest religious institutions in the world.  I will bet my life they have the resources to maintain those rooms well. They have the money, and God knows there is never a shortage of labor.

Why didn’t they do better? After all, they have the means. It has to be because they felt it was not necessary. The people who rent those rooms will gladly accept them as is.

Or, is it possible that the powers that be, think that the rooms, as they exist now, are indeed well maintained?

My mother, who lives in India, travels a lot, and almost surely has seen worse, thought the rooms were par for the course. Her sister, my aunt, who lives in Florida, had a major freakout. I, who travel between the U.S and India regularly, am a little bit more inured to the unevenness of Indian life, and staked out a middle ground – moderate irritation, tempered by grimly amused resignation.  There is a T-Shirt out there that says, “Surrendering to India”.  That is my mantra.

In fact, I don’t believe that the management really believes that the rooms are well-maintained. Everyone knows that more is possible.  You can find good maintenance in India. Maintenance where attention is paid to detail. Management that walks in the shoes of the guests, anticipates their needs, and satisfies them, efficiently, quietly, even imaginatively, and elegantly. In Tirumala itself, we found a restaurant, called Sarangi, which was a perfect example of this.

The difference between an establishment like Sarangi, and our rooms, is cost. That was an expensive restaurant. Only clientele of a certain economic class would frequent Sarangi.

So good maintenance, as defined above, is only available to the affluent. A very large swath of Indian society is never given good maintenance as the norm.  Not even when you have the means to provide it.  This slice of society accepts that lackluster maintenance is normal, and good enough. This is what they receive, and in turn, this is what they themselves are conditioned to provide.

You have a roof over your head, so what if the walls are stained and badly painted. You can close and lock the bath room door, so how does it matter if the door looks like it might give you a disease if you touch it. You have running water, so who cares if the water heater is mere ornamentation, and an ugly one at that.

The software you build performs the business function you need. So who cares if the user interface is confusing, error prone, and has poor finish.  The code runs, so who cares if it is unintelligible. The reported bug is fixed, so who cares if fix adds more strands of spaghetti.  The enhancement came in on time so who cares if you copied and pasted thousands of lines of code, rather than analyze, and re-factor.

Those inexpensive programmers that you outsource your work to – where do you think they learnt what is important and what is not? Where do you think they learn how to perform a task?   They learn it everyday, ever since they are in knee pants, in places like those guest rooms in Tirumala.

You see this everywhere.  Here is an another example.

There is a multiplex / food court /entertainment center, called Mayajaal, in Chennai. The multiplex has 16 screens. The tickets range from 120 rupees to 150 rupees I think. This place makes money hand over fist. Exactly like Tirumala, Mayajaal does not lack resources.

My parents and I went to a movie there, and took the elevator up to our screen. The linoleum floor of the elevator had a huge gash in it. Part of it was just gone. The railing that runs along the wall of the elevator had been ripped off. The restroom had those automatic air dryers, but the electrical sockets were hanging out of the walls. I mean there was a hole in the wall, and the board with the outlets had just been pushed into the hole.

I repeat, Mayajaal makes a lot of money. It makes enough to fix the elevator, and actually install the damn electrical outlets.

So why don’t they? Because neither Mayajaal, nor the folks that frequent Mayajaal, think it is truly necessary.

What do you believe these folks might think about “user experience”, or “software engineering”? I would guess, they consider such notions quaint, but ultimately unimportant. You get to see the movie, so who cares what the elevator looks like. You have a place to relieve yourself, so who cares how a fixture is installed.  God is not in the details. It simply is not part of their day to day life. They don’t see it. They don’t expect it. They don’t know how to supply it. They cannot understand how someone might think otherwise.

Many of your inexpensive programmers, are exactly these folks.

These past few years I’ve had occasion to work with such resources.   They were almost universally smarter, and more hard working than me.  They consistently delivered the goods in acceptable time.  However, I believe I consistently produced better software, soup to nuts – business and technical analysis, design (internal, and external), and construction.   The quality of their work made me very frustrated, angry, would leave me in despair, really.   I don’t think I will ever feel that way again.   You see, now I know where these folks are coming from.

How do you take a bright, enterprising, middle class, or lower middle class kid, from some small town like Khamman, or Villupuram, who has a degree from one of those hundreds of engineering colleges that have mushroomed up all over Andhra, and Tamil Nadu, pay him 2 or 3 lakhs a year, and get him to care about detail?   That is the challenge.   How do you inculcate a decidedly alien culture in someone like him?

A small thing that used to drive me, and some of my fellow travelers crazy, was the chronic misspelling of variable names in the code.   There is only one cure for it.  Visit India.  In the past few weeks, I’ve seen misspelt shop signs, and traffic sign boards.  My Dad’s letter-head has issues.  My cousin’s wedding invitation had an obvious error.  Note, these are all significant documents, affecting branding, and important life events.  Why did they not proof-read this stuff?   The money surely is available, and how much time does it take to proof-read a shop sign, or even a wedding invitation.

The answer – they don’t believe it matters.  How can we expect a programmer who grows up with this, to care about correctly spelling an identifier in computer code?   The user doesn’t even see it, for God’s sake.

My favorite so far is this road sign, on a bend in the Puthur – Oothukottai route from Tirupathi to Chennai: “Cures Ahead.  Goo Slow“.

Today, on my morning walk down Vandalur Road, I saw the local trash collection service in action.  They wore brown uniforms, and were dumping the trash into a metal cart, which they were pulling behind a bicycle.  Nice.   But.  The carts were open.   I could already see flies buzzing around, and in the cart.  Soon the cart would fill up, and trash would start to spill out.

Why did it not occur to them to put a lid on that small cart?   How are we going to get them to care about that missing lid?  I am coming around to the idea, that this is the primary challenge of outsourcing.  Everything else is just technical detail.

 

by

I want to create software that is like Tom Binh bags

or  I want to create software like Tom Binh makes bags.

I am impressed by Tom Binh bags.

It makes me happy to carry around a Tom Binh bag.

I am so glad that it has been possible for someone to make a success out of working in just that manner.

Craftsman

Is Tom Binh a craftsman?  Is that what it is?   Do I want to be a software craftsman?  That is possible, isn’t it?  There are people out there that see value in that, don’t they?

I have been doing computer programming of some sort or the other for almost 17 years.   I don’t feel very accomplished.  I feel a distinct lack of skill.  There are so many questions, which I don’t know the answer to.  I am not sure I am an expert at anything.  I know a little bit of a lot of things.  I can be relied on to ask good questions, I think.  I have been known to work hard.  I play well with other people.

At best, I think I have the aptitude to be a craftsman.  I keep thinking that you are a craftsman when the skill that you ply is fairly complete, and second nature.  I am not that, yet.  I have promise.  That is the most I can say for myself.

How can I work at something for 17 years, and still not be very good at it?  Are you ready for this?   It turns out that I don’t have to be.  In software engineering, this strange slice of the working world, I can provide a lot of value, without being very skilled at software engineering.  Despite my shortcomings, if software engineering skills were graded on a curve, I am sure I would fall at a decent spot.  I am good enough for a lot of environments.

Quality

When I see a Tom Binh bag, I think, “That is quality“.

A Tom Binh bag is an inspiration.   No, it is more.  A Tom Binh bag is an aspiration – for me, carrying one is an act of prayer.

But what am I aspiring to?  What does “quality” mean to me?

I am not sure I can explain it.   I could resort to that old saw about pornography – we can’t define it, but we sure know it when we see it.

Tom Binh’s bags show attention to detail.  “God is in the details“; sign me up for that church.

Every little part of the bag has a well thought-out purpose.  Every little part of the bag fits, as intended, into its surroundings, and all the little parts work together, as intended, to serve a larger, well-defined, goal.  Unmistakably, the bags are a rigorous architecture, and design, made flesh.

I want to create software that exhibits the same characteristics.  Is that too much to ask?  The answer of course, is yes.

Why don’t more people carry Tom Binh bags?

Good, but not worth the money

The bags cost more money than some folks want to, or can spend, on luggage.  Regardless of the value the bags provide, the finances just don’t make sense to these folks.

Software engineering shares this issue.   Higher skilled engineers are expensive.   Business, often does not want to spend that money.  The work produced by inexpensive, but low skilled engineers, is deemed good enough.

There is a school of thought that holds that a higher skilled software engineer is worth several low-skilled engineers.  Alas, I don’t know how to make this case, even though I have seen anecdotal evidence to support this idea.

This is not an entirely lost cause, but close.

I am looking, and I am not liking

Some people simply are never going to like Tom Binh bags.   Even when money is no object, they see nothing there to their liking.

Have you ever had to deal with a business stakeholder who wants a completely inexplicable UI?   The customer is always right.  You can only walk away from this.

Craftsmanship dictates low volumes?

I might be wrong about this.  Very little of Tom Binh’s production process seems to be mechanized.  Tom Binh is not in the mass manufacturing business, I think.  He creates quality with meticulous, manual, effort, by a small number of highly skilled folks.   Naturally, this means they can produce only so many bags.

Software does not have this limitation.  Mass manufacturing is trivial.   It is the master copy, however, which is the challenge.  Craftsmanship can indeed be brought to bear in creating and maintaining that one master.  However, we all know that is easier said than done.  Some days I understand why.  Some days I do not.

Colors

Oh man, do I love those colors!

tombinh-colors

I wonder, what color is the code I write?

What color is your code?

by

How to avoid sending null/empty fields between a RESTful service, and a client?

By it strictest definition, a RESTful service sends and receives ‘resources‘, which when seen from an object-oriented perspective, are largely the objects of your domain model.  An insurance policy, a bill, a payment, a car, a truck, a driver, a patient, a nurse, a clinic, etc.

Often, an instance of the object that you are putting on the wire has null fields.   Here is a patient at a clinic, whose middle name and date of birth, we don’t have.

{
       "patientId": "THX1138",
       "firstName": "John",
       "middleName": null,
       "lastName": Patient,
       "dateOfBirth": null,
       "address": "1,  Illness From,  Bankruptcy, HA-00000, USA"
}

In the mobile world, where we now pay for data usage, and where connections can be slow, it might be useful to minimize the amount of data we throw on the wire.   When the mobile app retrieves the above person, John Patient, I want to drop the fields (middle name, and date of birth) that are null, and send just this.

{
       "patientId": "THX113",
       "firstName": "John",
       "lastName": Patient,
       "address": "1,  Way Ticket To,  Bankruptcy, HA-00000, USA"
}

Further, I want the mobile app to be similarly discriminating when it sends data back to the RESTful service at the server.

In effect, the marshalling mechanism at both ends should be able to create the patient object with just the data that is available, and leave the rest of the fields null.

Javascript works that way natively.   If I was running Javascript at both endpoints, the core language itself would pretty much give us this behavior (Ah, node.js!).

I expect this is possible in other environments too.   In particular, I will need to look into this for the following.

  • RESTful service in Java, talking to 
    • Android App in Java
    • iOS App in Objective-C
    • Javascript MVC

68 dollar question

Is this really worth the trouble?   Will this really produce a lot of efficiency?

Is it even worth pursuing this question?

When data usage costs money, and connections can be slow, and intermittent, why would you want to carry unnecessary baggage?  Even in the best of circumstances, why would you want to?  This sounds like it ought to be a first principle for any mobile app.

But, we do want to watch out for ‘premature optimization’ – getting clever too soon, or even unnecessarily.

Hence, we will need to know how to test for this?  How do you simulate, and test spotty mobile connections?

Related questions

  • What other techniques exist for dealing minimizing the data usage of your mobile app?
  • What other techniques exist for dealing with intermittent, or slow connections?

These just seem a more severe variation of a problem that has been around for longer than the mobile world – how to build a UI around large-grained, stateless APIs.   So there are answers out there …..