March 09, 2020

Life Lessons : Growth

Every city has its own tales to talk about. Observing the people and the city itself can teach us valuable lessons for management and even life, giving new dimensions to life and self-realization that many books often fail to answer.

One such lesson answers the question :
  • How do you grow your business ?
  • What does it take to grow ?
  • Who do you call your competitor and how do you view them ?

Growth


Pune, Oxford of the East. Thousands of students from across the country and beyond come to the city in search for good education and better lifestyle. Next to one such management educational institute, on a busy street was my shack. "Amrut Chaiwala" as it was proudly called, in the name of my father “Amrutlal”. Right from the beginning I involved in the art of making tea. Yes, it's an art. Adding the right amount of ingredients, based on the quantity of tea you need to prepare, will the tea taste exactly the same, as always. Hundreds of people; students, teachers and others alike, visit our shop everyday and enjoy the refreshing chai and snacks. My father in his mid fifties, short, muscular man, works very hard brewing the tea, making snacks and serving happy customers. The best thing was hearing all the endless chitchat on topics ranging from politics, sports, economy to culture, management styles to gossips about the staff. 

After coming back from college, I used to come here and help father with the duties, happily listening to the gossips. Next to our place was a shabby little shed consisting mostly of a big platform and some tin sheets covering the sides, most of which were torn and broken. It was operated by a very old women and her grandson “Chotu”; couple of years younger than me, who offered vegetable upma ( a semolina based delicacy). They seemed very poor and even had no board or any other depiction denoting they offered snacks. Very few and specific people, who were regulars, even knew about it and ordered from there.

One hot summer evening, while father had gone out to buy some supplies, I was making the preparation for brewing tea. The street was bustling with noises of horns, yelling, engine roars and the chit-chat, gossips, laughter of the people around filling the entire space. While one eye was busy aiding the preparation of the tea, the other eye was gazing around to take any orders. I saw familiar group of people coming towards me and identified them as Ramesh and his friends. My eyes stopped to see Ramesh holding up his hand showing four fingers. A gentle nod and the order was placed. Four special tea along with four cream rolls. They are what we call Regulars. who visit almost everyday around the same time where just a glance is enough for communication. I saw him move to the next shed and order some upma’s.

Few months pass by and the world was gripped by recession. The chit-chat and gossips was now replaced with people throwing advice on economics to anyone who was listening. While the tea business was not as severely gripped in terms of daily footfall the rising cost of the materials and lower profit margins meant big impact on the savings. A few weeks earlier father had met with a near fatal accident and the treatment had already reduced the savings substantially. 
It was just another afternoon, a time when the business is pretty relaxed. There I saw Ramesh coming towards me with someone very senior to him. Giving order for 2 tea's he introduced me to the senior.

“Two special tea. This is my mentor, a famed economist and a family friend. Make the tea, special masala today.” said Ramesh.
On eyeing the next shed, which was in more miserable condition than earlier and seeing no one inside he enquired.
“Where is Chotu ?” gesturing to the next shed.

“He should be nearby, will be back in a minute or two”.

A few minutes later Chotu arrived at his shed, Ramesh went ahead and placed some order. Completing the tea and snacks, he went to the next shed, kept the plates and paid the bill. The economist seemed fairly impressed by the taste of the upma and was chatting with Chotu.

“This was one of the best upma’s I have tasted in a while.”

“Thank you Sir. Do visit again” replied Chotu.

“Why don’t you have a board which displays the menu ? It was really good and you should let the passerby’s know that you offer this wonderful food.” inquired the economist.

“We are very poor sir. We have no money to waste for such a board. People come to us have have tasty, delicious upma, not read boards or signs. We make limited quantity everyday knowing that we will be able to serve all our customers and where we know it will not be wasted.” replied Chotu

The economist was amazed by the simplicity and innocence of the young boy. He stated “Ok, but if you put up a board it will let other people, who are new here, know that you sell this fantastic food. Do you even know how to read and write ? Where are your parents ?”

“Yes Sir, I am currently in my first year of graduation. My parents died when I was very small. We were not in good shape back then. My grandmother took care of me, raised me, sent me to school to study. She actually setup this wonderful shed. She does not know how to read or write, maybe this might be the reason we have not added any board. But we don’t need it also. People come here, see others eating upma and know where to come.” replied Chotu smiling and feeling proud.

The economist was astonished by the unworldly attitude of the boy. 
He went on, “Ok, here’s the deal. You go and make a good board and put it up here”. The economist showed Chotu a perfect place for the board and continued. “I will be here next month, if you see no improvement in the business, I will give you the cost you incurred on the board. But if your business goes well you have to give me a plate for upma for free.”

Chotu seemed hesitant at first, but a nod from Ramesh seemed to strengthen the confidence and he finally agreed.

“You already make such fantastic upma, why don’t you start offering tea as well ?” the economist said.

“But sir, if we start selling tea, what will they sell ?” replied Chotu signalling to my shack.

The economist was shocked on hearing the reply. He never expected anyone to look at a competitor this way. We were competitors, atleast in his eyes. That is what the economist had believed. That is until now.

Chotu continued "They prepare tea so well that I will never be able to make it that way.”

The economist, seemingly now pondering over something, and Ramesh chatted for a few more minutes and left. 

A month passed by. Couple of weeks earlier, Chotu had put up a fresh looking yet classic styled board near the right top location, exactly where the economist had showed him. Slowly but surely the business had definitely increased. Instead of only the regulars, new faces were now seen ordering from his place. In fact that had helped my tea business grow as well. People would buy upma from there and have tea from here. 

Ramesh and the economist had visited us yesterday. The economist had called both Chotu and me aside and shared a plan. A plan of growth and prosperity. And the wheels of the plan were already set to motion. The economist had offered us some detailed insights on our business, given valuable advice and shared a master plan. We had never looked at each other as competitor’s, yet we had never considered each other as partners. But this was exactly what was his master plan. 

The economist had said “You both are the best at what you do. One makes the best, most refreshing tea while the other offers the most delicious, tastiest snack. One offers a way to satisfy the hunger while the other offers a way to reduce the stress. One offers a way to fill the tummy to satisfaction while the other offers a way to relax in this unrelenting world. But both of you are missing the other important piece when serving individually. You already have observed how impact on one of the business will affect another. The customer is looking for obviously good and delicious food but if two similar options are available, he will naturally be attracted and choose one which offers better customer experience. You both have the first part covered. The second part though can be improved. Think about it. Working together will also increase the customer engagement as now there are more hands to serve and look after the customer.”

Ramesh was also pitching in from time to time. He went on for nearly 45 mins and patiently explained all the queries we had. One last thing that I was unclear about and finally asked was “What's in it for you ?”

The economist smiled and breathe a sigh of relief. He realised that we both were more in acceptance of the idea. Finally he replied “You can offer me free food every time I visit, if you want”.

We all laughed and continued the discussion.

Few years went by. The encouraging words of wisdom from the economist had done its magic. Not only did our business improve but so did our lives. We will finally be moving from the little shack to a proper shop we had purchased in a building nearby. We also started offering more and different varieties of snacks and other beverages. We had unlocked the secret to unlimited growth. 

---

Sometimes growth can be achieved by going in unexpected directions.


August 20, 2019

API Mocking / Simulation Tool Comparison

So we have seen Hoverfly and Wiremock API Simulations tools earlier and looked at Java examples of using them. Lets look at how both of them compare on various parameters and which one is best fit for your use-case.
We will do a comparison based on the few parameters as below and look at limitations and advantages over each other
  • Support for Capture Replay : Ability to capture the requests once and replay them during simulation
  • Support for HTTPS : Support for Https based API's
  • Client Side required changes : Changes required at the client, if any
  • Support for Stateful API's : Support for states when using stateful API's
  • Support for Response Templating : Ability to templatize responses to give dynamic behavior
  • Miscellaneous : Other miscellaneous parameters and unique features

Hoverfly

Hoverfly released under Apache 2.0 license primarily written using Go language is developed and maintained by SpectoLabs.

Support for Capture Replay

Capture is only supported in Proxy mode and is not supported in Web-Server mode. The captured requests can be exported to a Json file. Hoverfly has clear, defined and documented Simulation Schema.

Support for HTTPS

Proxy mode has the support for capture, simulate HTTPS API's and can use custom generated certificates for authentication. The Web-Server mode does not support HTTPS.

As it acts as forward proxy HTTPS is directly to and fro actual end-point. It also requires client to support the HTTP protocol to make the initial connection.

Client Side Changes

The client used should support and honor the 'Proxy Settings' that needs to be configured for using the tool. As the capture works in forward proxy no changes are required on the client side.

Support for Stateful API's

Hoverfly supports simulation of stateful API's. You can define states and give each requests a current state for matching and next transition state to move to the new state.

Support for Response Templating

Hoverfly supports response templatization to build dynamic responses. It comes with few predefined helper templates and uses a library based on handlebars for templating.

Miscellaneous

  • One of the main features of hoverfly is use of forward proxy. It allows to run in what is called as SPY mode where few API call can be simulated while the non-matching ones can call the real API.
  • Supports multiple endpoints for capture/simulation.
  • Has support for importing/resetting simulations at runtime. 
  • Ability to capture all the headers or specific mentioned headers
  • Provides Python and Java bindings
  • Allows Man in the Middle behavior for both HTTP and HTTPS through the use of middleware mode
  • Documented Simulation Schema 

Wiremock

Wiremock also released under Apache 2.0 license is written in Java, developed and maintained by Tom Akehurst

Support for Capture Replay

Capture is supported and there is the old (legacy) as well as the new way to capture the requests. Captured requests can be exported in Json file. Each request is exported to a separate file.

Support for HTTPS

Wiremock provides support for HTTPS and can use custom generated certificates for authentication. As it acts as reverse-proxy the actual end point is HTTPS details are hidden. The wire-mock server acts as the endpoint for the client.

Client side changes

The endpoint should be changed to point to wiremock server which can act as the reverse proxy for the actual endpoint during the capturing of requests.

Support for Stateful API's

Stateful API's are supported through Scenarios. You can define Scenario names and the different states in the scenarios which the request will match and transition during simulation. The legacy method of capture does not support stateful capture.

Support for Response Templating

Templatization is supported through the handlebars library. It contains the string and other helpers provided by the Java implementation of handlebars by jknack

Miscellaneous

  • Due to reverse proxy nature the clients are independent of the protocol of the actual endpoint
  • Has support for forward-proxy via enable-browser-proxy

Final Thoughts

So now that we have looked into both the tools lets look at what are few advantages and limitations over each other. The list is in no way complete but can act as a quick guide to identify which tools better fits your use-case.

Hoverfly Advantages

  • Support for SPY Mode where few API's are simulated while others are passed to the real endpoint
  • Defined and Documented Simulation Schema
  • Run-time importing/resetting of the simulations
  • Supports multiple endpoint with single instance (at-least in the proxy mode)

Hoverfly Limitation

  • Cannot capture requests in webserver mode
  • Web-server mode does not support HTTPS
  • Dependent on the client for support and use of system/manual proxy settings

Wiremock Advantages

  • Independent of the protocol of the actual endpoint
  • No configuration/changes required by the client.

Wiremock Limitations

  • Runtime import/resetting of simulations not supported. Simulation mappings needs to be defined at the start during creation/initialization.
  • Cannot capture all headers. Needs a list of the headers that needs to be captured.
  • Simulation Schema though defined is not documented completely anywhere.
  • Only one endpoint can be used with one instance. (Capture needs to be specified with a target url).

July 19, 2019

HTTP API Simulation using Wiremock


In today's world of micro-services we typically integrate to many REST API (working on HTTP or HTTPS) for our application. We already saw one such simulation tool called Hoverfly.

Wiremock is another similar tool for simulation of the API's. Take a look at the documentation of the tool. It supports both standalone (or command-line) mode as well as Java API.

Lets look at a sample API simulation example using Wiremock with Java.

We will use maven to create a new application and add the following dependency
<dependency>
    <groupId>com.github.tomakehurst</groupId>
    <artifactId>wiremock-jre8</artifactId>
    <version>2.23.2</version>
</dependency>

Once the dependencies are installed we want to instantiate and start WireMock as below :
WireMockServer wireMockServer = new WireMockServer(WireMockConfiguration.wireMockConfig()
//        .dynamicPort()
//        .usingFilesUnderDirectory("~/simulation/")
          .port(8888));

wireMockServer.getOptions().filesRoot().child(WireMockApp.MAPPINGS_ROOT).createIfNecessary();
wireMockServer.getOptions().filesRoot().child(WireMockApp.FILES_ROOT).createIfNecessary();

wireMockServer.start();

Note that above before starting the server we actually create the necessary directories where the simulation files will get stored. They are stored under 'mappings' (as given by MAPPING_ROOT) and '__files' (as given by FILES_ROOT) under the root directory. The default root directory is 'src/test/resources'.

Once the server is started we want to start recording and capture the requests. This can be done as below :
// Start Recording the request and proxy all the requests to http://echo.jsontest.com
wireMockServer.startRecording("http://echo.jsontest.com/");

// Create a client to call the echo url. Note that we are calling 
// the wiremock server which will act as a proxy to the configured url
HttpClient client = HttpClientFactory.createClient();
HttpGet request = new HttpGet("https://localhost:8888/a/b/c/d");
HttpResponse response = client.execute(request);

System.out.println("Response Code : " + response.getStatusLine().getStatusCode());
BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
StringBuffer result = new StringBuffer();
String line = "";
while ((line = rd.readLine()) != null) {
    result.append(line);
}
System.out.println(result.toString());

// Stop the recording. The recorded mapping are stored inside the root directory
// and returned as result
SnapshotRecordResult recordedMappings = wireMockServer.stopRecording();

So now we have captured the recording it is time to test the simulated response. Here the recording is stopped but the wiremock server is still running. It will serve all the captured responses for similar requests.
Lets try it out
// We call the same URL and this time we will get simulated response
HttpGet same_request = new HttpGet("https://localhost:8888/a/b/c/d");
HttpResponse simulated_response = client.execute(same_request);
System.out.println("Response Code : " + simulated_response.getStatusLine().getStatusCode());
BufferedReader simBR = new BufferedReader(new InputStreamReader(simulated_response.getEntity().getContent()));
StringBuffer simBuffer = new StringBuffer();
line = "";
while ((line = simBR.readLine()) != null) {
    simBuffer.append(line);
}
System.out.println(simBuffer.toString());

// We call a different URL. As this is not captured it will return Not Found in the response
HttpGet different_request = new HttpGet("https://localhost:8888/e/f/g/h");
HttpResponse no_response = client.execute(different_request);
System.out.println("Response Code : " + no_response.getStatusLine().getStatusCode());
BufferedReader noBR = new BufferedReader(new InputStreamReader(no_response.getEntity().getContent()));
StringBuffer noBuffer = new StringBuffer();
line = "";
while ((line = noBR.readLine()) != null) {
    noBuffer.append(line);
}
System.out.println(noBuffer.toString());

At the end we need to stop the wiremock server as below
wireMockServer.stop();

Above we used record-replay to simulate the endpoints. Wiremock also provides methods for creating custom responses by using stub().

So happy simulating :)


June 29, 2019

Mocking API Requests using Hoverfly Java

 In today's world of micro-services we typically integrate to many REST API (working on HTTP or HTTPS) for our application. For development and testing we need an environment to work with those API. For this typically we write stubs or create a test-web-server to simulate those API. 

Hoverfly is one such tool that allows us to have API simulation. Take a look at the documentation of the tool in the standalone (or command-line) mode (Check out the basic tutorial). Its pretty straight forward and simple.

For development it may be feasible to use the command-line tool, but for the automated tests in the build process having this setup may be a bit of effort to setup the environment and use it. 
Instead Hoverfly also has Java (and Python) bindings for the tool that we can use in the automation.

Lets look at a sample mock API example using Java.

We will be using maven to create a new application and add the following dependency

 <dependency>
     <groupId>io.specto</groupId>
     <artifactId>hoverfly-java</artifactId>
     <version>0.12.0</version>
 </dependency>

Also make sure you are setting the Java compiler version in the properties

<properties>
    <!-- other properties, if any -->
    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.target>1.8</maven.compiler.target>
</properties>

Now lets use hoverfly to capture and store the requests :
// Initialize Hoverfly instance
try (Hoverfly hoverfly = new Hoverfly(HoverflyConfig.localConfigs().captureHeaders("Content-Type", "Date"), HoverflyMode.CAPTURE)) {
    // Start hover-fly
    hoverfly.start();
    
    // Create a client to execute http requests. OkHttpClient comes bundled with Hoverfly. You can use any http-client.
    OkHttpClient client = new OkHttpClient();
    Request dateReq = new Request.Builder().url("http://date.jsontest.com").build();
    
    try (Response response = client.newCall(dateReq).execute()) {
        System.out.println("REAL : " + response.body().string());
    } catch (IOException e) {
        e.printStackTrace();
    }
    
    // Save/Export the response to a file
    hoverfly.exportSimulation(Paths.get(System.getProperty("user.dir"), "capture_requests.json"));
}
You can check the capture_requests.json to view the request-response that got saved.
Now lets simulate the request
// Initialize Hoverfly in what they call SPY mode
try (Hoverfly hoverfly = new Hoverfly(HoverflyConfig.localConfigs(), HoverflyMode.SPY)) {
    hoverfly.start();
    
    // Initialize client to making http request
    OkHttpClient client = new OkHttpClient();
    
    // As we have not yet imported any simulation, we will get the response from the real API call
    Request timeReq = new Request.Builder().url("http://time.jsontest.com").build();
    Request dateReq = new Request.Builder().url("http://date.jsontest.com").build();

    try (Response response = client.newCall(timeReq).execute()) {
        System.out.println("REAL : " + response.body().string());
    }
    
    try (Response response = client.newCall(dateReq).execute()) {
        System.out.println("REAL : " + response.body().string());
    }
    
    // Now lets import the simulation.
    // Remember we had captured 'date.jsontest' request which will give simulated responss. Any other requests
    // will still call the real API and give the response
    hoverfly.simulate(SimulationSource.file(Paths.get(System.getProperty("user.dir"), "capture_requests.json")));
    
    try (Response response = client.newCall(timeReq).execute()) {
        System.out.println("REAL : " + response.body().string());
    }
    
    try (Response response = client.newCall(dateReq).execute()) {
        System.out.println("SIMULATED : " + response.body().string());
    }
} 
Above we used captured request simulation but you can easily build your own simulation quickly using what they call 'dsl'
So lets start Simulating :)

June 29, 2017

[Windows]: Executing a script on Windows startup

If anyone has used Linux and have been working on how to have a script executed at machine startup, that person will tell you how simple it is to achieve it.
Having written this myself for a Linux based server where the script performed startup of all required components for my project and did the necessary initialization sequence and other stuff.
Along came a client who had a Windows 2012 server and I was tasked with replicating the same as the Linux box on Windows. So I started writing a Powershell script to do the startup sequence of my project. Well, it was not as smooth and simple task (as was with Linux) to write a similar Powershell script but finally after a few brain damaging moments (pun intended) I finally managed to complete the script.

So only the final step pending was to have the script execute on machine boot. Well, this step sounded really simple until it hit me hard and after literally banging my head against the walls
I finally managed to complete it *correctly*. While there are blogs/tutorials giving a guide for the same it was really useless because I wasn't getting my script to work. After much research and some more head-bangs-against-the-wall the below steps finally worked :
  1. Press the Windows key and type in "Task Scheduler"
  2. Once the result is found right-click the entry and click "Run as administrator". If it is not started as Administrator, the script which we are about to specify will not run correctly. (This was one of the head-bang moment)
  3. Once the application opens, click "Create Task.." as shown in the image
  4. Fill in the details :
    • Give Name and Description as appropriate
    • Select the User Account with which it should run
    • Select "Run whether user is logged on or not."
    • Check Run with highest privileges.
    • Check Hidden
    • Provide "Configure for" as the current OS version which this will run on.
  5. Goto the Trigger Tab
    • Click New
    • Specify the necessary Trigger details as appropriate
      • For executing on machine reboot select "At Startup"
      • Click Ok
  6. Goto Action Tab
    • Click New
    • For the Action select "Start a program"
    • Since the Task Scheduler will run the script as a normal batch script and not a powershell script. To execute a powershell script in the Program/Script specify the path to powershell executable. 
    • This is typically "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe"
    • Add the arguments as "-executionpolicy bypass .\{{script-filename}}.ps1 > {{output-log}}.txt 2>&1"
    • Add "Start In" as the directory where the script is located
    • Now the arguments deserves some explanation :
      • Execution Policy specifies how/if should the execution begin. This is more related to using Powershell and the way Windows System handles execution. This is way too complex and you can read more on this here if you want (or go the simple route of specifying it as bypass).
      • Next is the script file name but notice we are giving relative path. You can also optionally give absolute path.
      • Next is the redirection of the output and/or we also redirect the errors to the same output
      • Note : here that it is different from the other guides as we do not specify the "-file" option. This was the head-band-multiple-times-against-the-wall moment because that is when I realized that giving the -file options (among the many other issues also due to this) is the reason why the redirection of the output and errors was not working.
    • Click Ok
  7. Click on the Conditions Tab
    • Change the settings as appropriate 
  8. Click on the Setting Tab
    • Change the settings as appropriate 
  9. Click OK
  10. And we are done :)
The output logs, error logs really helped me iron out the few issues that still were present in the script. But to get that working was a real strange mystery (and finally realizing the "-file" option as the culprit was a bummer).

Thanks and have a good day :)

      June 06, 2017

      Moto Vs Xiaomi

      This is more of a comparison based on personal experience between Motorola and Xiaomi phones.

      Having used Motorola phone for the past 2+ years (Moto G2 and Moto G Turbo), recently I made a switch to Xiaomi Redmi Note 4, and yes, the 64 GB/4GB version is what I am talking about.

      Although not very keen on changing the phone as I pretty much loved the Moto G Turbo, I ditched Moto for the hype of 4 GB RAM (My laptop has less RAM than this !) with a huge storage space and exciting price.

      While I say that the Redmi is a powerful phone (atleast on paper :)) and the build quality is good, still nothing can beat the simplicity and smoothness of the Moto.

      Lets compare the Redmi Note 4 with Moto G Turbo :

      Specs and Build Quality
      To simply put, build quality of both the phones is superb.
      On paper the Redmi Note 4 wins hands down on the specs from Moto G Turbo, with 5.5 inch screen vs 5 inch, 4GB Ram vs 2GB Ram, 4000mAh battery vs 2500mAh and so on.
      Redmi supports Fingerprint scanner while the Moto is IP67 certified making it water and dust proof.

      What Redmi wins on the hardware front, it kind of loses on the phone build and looks. With Moto G Turbo you get a superb back cover which really helps get a grip of the phone in the hand, on the other hand Redmi's back cover is super smooth and the phone is bound to slip from the hands making it difficult to use (unless you use a case cover). Moto also has added a subtle touch added on the power/wake button making it feel rugged and easier to recognize. So on the phone build and looks its Moto hands down.

      Software
      Redmi Note 4 comes with Android 6 (Marshmallow) with MIUI skin on top of Android. On the other hand Moto G Turbo comes with vanilla Android 6 (Marshmallow), upgradable to Android N. While MIUI has been praised for its ability to customize almost everything, vanilla android stands out for its simplicity, beauty, and common-sense. The vanilla android has the omni-present Google Bar where you can search almost anything. The Google Now integration on the Moto is also much smoother all its takes is a left swipe to view all the relevant information right at your fingertips. The MIUI skin also comes bloated with too many apps and softwares. Though I really liked MIUI for its three-finger swipe to take screenshot and the messaging app where it detects if OTP, and other codes and lets you copy it with a click.

      Camera
      Both the phones Camera quality is very good. The Note 4 camera app lets you have more control and comes with multiple options for filters, but its all software which can be achieved on the Moto G Turbo by installing the right app. The picture quality from both the phones is pretty much decent.

      Conclusion

      Redmi Note 4 is a powerful phone and there is no doubt about it. But it is also bloated and feels sluggish under heavy use through multiple apps. Make no mistake if there is only one app running it runs smoothly, but starting Whatsapp to read the messages, replying to an email, reading the news/weather updates, and listening to music !, well that takes a toll on the Redmi. With Moto on the other hand the experience is as smooth as butter, though some extremely high graphics game do take a toll.
      So its Moto for general, day-to-day use and with its super smooth Android experience, Moto integration apps, and the like its the best bet on buying a phone. And with latest Moto models having fingerprint scanner and the likes its seems hard to resist.
      Redmi does have its appreciation for its value-for-money, but until the software is fixed, I recommend only for intensive gamers and the likes.