April 14, 2021

Distributed Tracing : What is the "cost" of instrumentation ?

Distributed Tracing

Ok, what is hell is "distributed tracing" ?

Microservices Architecture is on the rise and is extensively used to power applications and services that we use on a daily basis.  Netflix, Amazon, eBay, to name a few, are based on microservice architecture.

With micro-service architecture a user request will typically span multiple services across different servers before stitching the response and sending it back to the user. The problem with this is monitoring, debug-ability, reduced global visibility.

Distributed tracing, also called distributed request tracing, is a method used to profile and monitor applications, especially those built using a microservices architecture. It refers to methods of observing requests as they propagate through distributed systems. It’s a diagnostic technique that reveals how a set of services coordinate to handle individual user requests. Distributed tracing requires that software developers add instrumentation to the code of an application.

OpenTracing provides API specification allowing to add instrumentation to the application code in a vendor neutral manner.

Cost Of Instrumentation

Services would usually talk to each other through some sort of IPC. There are many frameworks to allow such communication. Many frameworks provide inbuilt support for instrumentation, making it simple to enable distributed tracing. These framework usage and performance are studied extensively. The OpenTracing Blog is probably a good place to start.

So now let's come to the significant question "what is the cost of instrumentation" ?
What is the performance impact of adding instrumentation to the application code ?

For measuring the cost we will use the jaeger tracer and JMH to capture the metrics.

JMH provides an API to consume cpu cycles varying linearly with token value specified. We will use this to mock a long running job which will be instrumented.

// We need to consume or return the result to avoid JVM dead code optimization
public long processLongJob(long token) {
Blackhole.consumeCPU(token);
return token;
}

 

We will measure the metrics without any instrumentation, with NoOpTracer and JaegerTracer with default initialization values.

 

Benchmark

Param: token

AverageTime (ns/ops)

Error (99.9%)

Cost (Impact %)

NoInstrumentation

1000

1664.84177

49.940273

0

NoInstrumentation

5000

8324.19976

107.233129

0

NoInstrumentation

10000

16606.7095

60.983405

0

NoInstrumentation

50000

83249.7561

2113.50206

0

NoInstrumentation

100000

166090.738

1693.40512

0

NoInstrumentation

500000

829908.745

6188.7579

0

 

 

 

 

 

NoOpTracer

1000

1648.86056

4.521151

-0.960

NoOpTracer

5000

8280.19528

86.68164

-0.529

NoOpTracer

10000

16569.8697

134.783752

-0.222

NoOpTracer

50000

83254.7547

645.070768

0.006

NoOpTracer

100000

166211.527

979.291791

0.073

NoOpTracer

500000

833341.912

14473.6203

0.414

 

 

 

 

 

JaegerTracer

1000

2373.87521

2.188993

42.589

JaegerTracer

5000

10059.0192

18.01787

20.841

JaegerTracer

10000

18803.8006

63.222248

13.230

JaegerTracer

50000

87511.7573

651.314285

5.120

JaegerTracer

100000

172132.755

959.72657

3.638

JaegerTracer

500000

846543.298

8904.63822

2.004

As seen above No Instrumentation and NoOpTracer scores are comparable with virtually no impact on performance while JaegerTracer costs ~1.5x but decreases with increase in CPU cycles consumed.

Lets look at the average time per fixed-CPU-cycles. As CPU cycles consumed varies linearly with the token value we can divide the time by the token. Below table summarizes the above table into scores for the different instrumentation techniques


Token

Score (AverageTime / Token)

NoInstrumentation Score

NoOpTracer Score

JaegerTracer Score

1000

1.664841774

1.648860562

2.373875211

5000

1.664839952

1.656039056

2.011803834

10000

1.660670949

1.656986968

1.880380062

50000

1.664995121

1.665095094

1.750235147

100000

1.660907376

1.662115265

1.721327553

500000

1.65981749

1.666683825

1.693086597

Plotting the table with token on X Axis and the scores on Y axis it becomes clear that the more intensive the code being instrumented the lesser it has impact on performance.

Using the above measurements instrumenting any function whose total execution time > 1 ms, the cost of instrumentation is negligible.

That's all folks. Till next time.

April 27, 2020

Thanos is Coming ! Is Everyone Ready ?

On this day (27th April) two years back with the release of Avengers: Infinity Wars, Dr. Banner had crashed landed in the New York City Sanctum warning everyone about the Thanos. The Mad Titan.

"Thanos is Coming. He is coming” he had warned.

In the current reality as well, Thanos is coming. He is coming. Dr. Banner has already warned everyone, but no one is listening to him. And this time Thanos‘es plans are different. Instead of wiping half of all living things, he is here to put an end to the mankind species for good ! That way he can get rid of anyone who can stop his plans. Plans for eliminating half the universe and ending all the sorrows and pain. And so he thinks.
 
Thanos is currently the most powerful being in the universe. Kind of like manifestation of God. And he currently holds three of the six Infinity Stones. Space. Power. And Reality.

Now the question remains.

Where are the Avengers and what are they doing to stop Thanos ?
And where are the rest of the Infinity Stones ?

In the MCU the Avengers are group of superheroes from across the universe. In the current reality universe, each and every human being is a super hero. There is an Avenger in each and every one of us. But we have buried the superhero deep inside within us, crushed under the weight of all the hatred, anger, ego, negativity.

And the Avenger within us holds the other three of the Infinity Stones. Mind. Time. And Soul. Few of us have already seen or used the Mind Stone (Or Intelligence). When used in a good way, it has led humanity forward by giving people access to best healthcare or working to eradicate hunger, to name a few. While there are those who have misused the intelligence to create unrest using weapons of violence and crime. We also possess the Gift Of Life given by the Time Stone. Everyone has one lifetime worth of time to live a beautiful and magical life.

The sixth infinity stone, the Soul Stone resides deep inside the heart of every Avenger, guarded by soul-keeper, who only allows someone to access the stone at a cost. And the cost is devotion. Devotion of your life towards the betterment of the people and the universe.

Thanos The Mad Titan, has harnessed the power of the stones, to become the all powerful being with ability to control the space, power and reality.
In order to defeat Thanos, mankind must work together, using all the three stones we possess. Mind or Intelligence, Time or Life and Soul or Heart. 

Use your time wisely, train your mind effectively and devote your soul completely. Use the power of the stones towards making the world a better place.

Few recent events are an indication, that children of Thanos have arrived on earth to steal the stones from us. And, together, we have defeated them, each and every single time. Slowly but surely, the imminent arrival of Thanos nears. One can never be sure when he may arrive. But we can be sure that we can defeat Thanos. Not individually, but together. Using the power of the three stones that we possess.

Remember, there is an Avenger in each an every one of you. Bring out the super hero in you and change the world.

March 09, 2020

Life Lessons : Growth

Every city has its own tales to talk about. Observing the people and the city itself can teach us valuable lessons for management and even life, giving new dimensions to life and self-realization that many books often fail to answer.

One such lesson answers the question :
  • How do you grow your business ?
  • What does it take to grow ?
  • Who do you call your competitor and how do you view them ?

Growth


Pune, Oxford of the East. Thousands of students from across the country and beyond come to the city in search for good education and better lifestyle. Next to one such management educational institute, on a busy street was my shack. "Amrut Chaiwala" as it was proudly called, in the name of my father “Amrutlal”. Right from the beginning I involved in the art of making tea. Yes, it's an art. Adding the right amount of ingredients, based on the quantity of tea you need to prepare, will the tea taste exactly the same, as always. Hundreds of people; students, teachers and others alike, visit our shop everyday and enjoy the refreshing chai and snacks. My father in his mid fifties, short, muscular man, works very hard brewing the tea, making snacks and serving happy customers. The best thing was hearing all the endless chitchat on topics ranging from politics, sports, economy to culture, management styles to gossips about the staff. 

After coming back from college, I used to come here and help father with the duties, happily listening to the gossips. Next to our place was a shabby little shed consisting mostly of a big platform and some tin sheets covering the sides, most of which were torn and broken. It was operated by a very old women and her grandson “Chotu”; couple of years younger than me, who offered vegetable upma ( a semolina based delicacy). They seemed very poor and even had no board or any other depiction denoting they offered snacks. Very few and specific people, who were regulars, even knew about it and ordered from there.

One hot summer evening, while father had gone out to buy some supplies, I was making the preparation for brewing tea. The street was bustling with noises of horns, yelling, engine roars and the chit-chat, gossips, laughter of the people around filling the entire space. While one eye was busy aiding the preparation of the tea, the other eye was gazing around to take any orders. I saw familiar group of people coming towards me and identified them as Ramesh and his friends. My eyes stopped to see Ramesh holding up his hand showing four fingers. A gentle nod and the order was placed. Four special tea along with four cream rolls. They are what we call Regulars. who visit almost everyday around the same time where just a glance is enough for communication. I saw him move to the next shed and order some upma’s.

Few months pass by and the world was gripped by recession. The chit-chat and gossips was now replaced with people throwing advice on economics to anyone who was listening. While the tea business was not as severely gripped in terms of daily footfall the rising cost of the materials and lower profit margins meant big impact on the savings. A few weeks earlier father had met with a near fatal accident and the treatment had already reduced the savings substantially. 
It was just another afternoon, a time when the business is pretty relaxed. There I saw Ramesh coming towards me with someone very senior to him. Giving order for 2 tea's he introduced me to the senior.

“Two special tea. This is my mentor, a famed economist and a family friend. Make the tea, special masala today.” said Ramesh.
On eyeing the next shed, which was in more miserable condition than earlier and seeing no one inside he enquired.
“Where is Chotu ?” gesturing to the next shed.

“He should be nearby, will be back in a minute or two”.

A few minutes later Chotu arrived at his shed, Ramesh went ahead and placed some order. Completing the tea and snacks, he went to the next shed, kept the plates and paid the bill. The economist seemed fairly impressed by the taste of the upma and was chatting with Chotu.

“This was one of the best upma’s I have tasted in a while.”

“Thank you Sir. Do visit again” replied Chotu.

“Why don’t you have a board which displays the menu ? It was really good and you should let the passerby’s know that you offer this wonderful food.” inquired the economist.

“We are very poor sir. We have no money to waste for such a board. People come to us have have tasty, delicious upma, not read boards or signs. We make limited quantity everyday knowing that we will be able to serve all our customers and where we know it will not be wasted.” replied Chotu

The economist was amazed by the simplicity and innocence of the young boy. He stated “Ok, but if you put up a board it will let other people, who are new here, know that you sell this fantastic food. Do you even know how to read and write ? Where are your parents ?”

“Yes Sir, I am currently in my first year of graduation. My parents died when I was very small. We were not in good shape back then. My grandmother took care of me, raised me, sent me to school to study. She actually setup this wonderful shed. She does not know how to read or write, maybe this might be the reason we have not added any board. But we don’t need it also. People come here, see others eating upma and know where to come.” replied Chotu smiling and feeling proud.

The economist was astonished by the unworldly attitude of the boy. 
He went on, “Ok, here’s the deal. You go and make a good board and put it up here”. The economist showed Chotu a perfect place for the board and continued. “I will be here next month, if you see no improvement in the business, I will give you the cost you incurred on the board. But if your business goes well you have to give me a plate for upma for free.”

Chotu seemed hesitant at first, but a nod from Ramesh seemed to strengthen the confidence and he finally agreed.

“You already make such fantastic upma, why don’t you start offering tea as well ?” the economist said.

“But sir, if we start selling tea, what will they sell ?” replied Chotu signalling to my shack.

The economist was shocked on hearing the reply. He never expected anyone to look at a competitor this way. We were competitors, atleast in his eyes. That is what the economist had believed. That is until now.

Chotu continued "They prepare tea so well that I will never be able to make it that way.”

The economist, seemingly now pondering over something, and Ramesh chatted for a few more minutes and left. 

A month passed by. Couple of weeks earlier, Chotu had put up a fresh looking yet classic styled board near the right top location, exactly where the economist had showed him. Slowly but surely the business had definitely increased. Instead of only the regulars, new faces were now seen ordering from his place. In fact that had helped my tea business grow as well. People would buy upma from there and have tea from here. 

Ramesh and the economist had visited us yesterday. The economist had called both Chotu and me aside and shared a plan. A plan of growth and prosperity. And the wheels of the plan were already set to motion. The economist had offered us some detailed insights on our business, given valuable advice and shared a master plan. We had never looked at each other as competitor’s, yet we had never considered each other as partners. But this was exactly what was his master plan. 

The economist had said “You both are the best at what you do. One makes the best, most refreshing tea while the other offers the most delicious, tastiest snack. One offers a way to satisfy the hunger while the other offers a way to reduce the stress. One offers a way to fill the tummy to satisfaction while the other offers a way to relax in this unrelenting world. But both of you are missing the other important piece when serving individually. You already have observed how impact on one of the business will affect another. The customer is looking for obviously good and delicious food but if two similar options are available, he will naturally be attracted and choose one which offers better customer experience. You both have the first part covered. The second part though can be improved. Think about it. Working together will also increase the customer engagement as now there are more hands to serve and look after the customer.”

Ramesh was also pitching in from time to time. He went on for nearly 45 mins and patiently explained all the queries we had. One last thing that I was unclear about and finally asked was “What's in it for you ?”

The economist smiled and breathe a sigh of relief. He realised that we both were more in acceptance of the idea. Finally he replied “You can offer me free food every time I visit, if you want”.

We all laughed and continued the discussion.

Few years went by. The encouraging words of wisdom from the economist had done its magic. Not only did our business improve but so did our lives. We will finally be moving from the little shack to a proper shop we had purchased in a building nearby. We also started offering more and different varieties of snacks and other beverages. We had unlocked the secret to unlimited growth. 

---

Sometimes growth can be achieved by going in unexpected directions.


August 20, 2019

API Mocking / Simulation Tool Comparison

So we have seen Hoverfly and Wiremock API Simulations tools earlier and looked at Java examples of using them. Lets look at how both of them compare on various parameters and which one is best fit for your use-case.
We will do a comparison based on the few parameters as below and look at limitations and advantages over each other
  • Support for Capture Replay : Ability to capture the requests once and replay them during simulation
  • Support for HTTPS : Support for Https based API's
  • Client Side required changes : Changes required at the client, if any
  • Support for Stateful API's : Support for states when using stateful API's
  • Support for Response Templating : Ability to templatize responses to give dynamic behavior
  • Miscellaneous : Other miscellaneous parameters and unique features

Hoverfly

Hoverfly released under Apache 2.0 license primarily written using Go language is developed and maintained by SpectoLabs.

Support for Capture Replay

Capture is only supported in Proxy mode and is not supported in Web-Server mode. The captured requests can be exported to a Json file. Hoverfly has clear, defined and documented Simulation Schema.

Support for HTTPS

Proxy mode has the support for capture, simulate HTTPS API's and can use custom generated certificates for authentication. The Web-Server mode does not support HTTPS.

As it acts as forward proxy HTTPS is directly to and fro actual end-point. It also requires client to support the HTTP protocol to make the initial connection.

Client Side Changes

The client used should support and honor the 'Proxy Settings' that needs to be configured for using the tool. As the capture works in forward proxy no changes are required on the client side.

Support for Stateful API's

Hoverfly supports simulation of stateful API's. You can define states and give each requests a current state for matching and next transition state to move to the new state.

Support for Response Templating

Hoverfly supports response templatization to build dynamic responses. It comes with few predefined helper templates and uses a library based on handlebars for templating.

Miscellaneous

  • One of the main features of hoverfly is use of forward proxy. It allows to run in what is called as SPY mode where few API call can be simulated while the non-matching ones can call the real API.
  • Supports multiple endpoints for capture/simulation.
  • Has support for importing/resetting simulations at runtime. 
  • Ability to capture all the headers or specific mentioned headers
  • Provides Python and Java bindings
  • Allows Man in the Middle behavior for both HTTP and HTTPS through the use of middleware mode
  • Documented Simulation Schema 

Wiremock

Wiremock also released under Apache 2.0 license is written in Java, developed and maintained by Tom Akehurst

Support for Capture Replay

Capture is supported and there is the old (legacy) as well as the new way to capture the requests. Captured requests can be exported in Json file. Each request is exported to a separate file.

Support for HTTPS

Wiremock provides support for HTTPS and can use custom generated certificates for authentication. As it acts as reverse-proxy the actual end point is HTTPS details are hidden. The wire-mock server acts as the endpoint for the client.

Client side changes

The endpoint should be changed to point to wiremock server which can act as the reverse proxy for the actual endpoint during the capturing of requests.

Support for Stateful API's

Stateful API's are supported through Scenarios. You can define Scenario names and the different states in the scenarios which the request will match and transition during simulation. The legacy method of capture does not support stateful capture.

Support for Response Templating

Templatization is supported through the handlebars library. It contains the string and other helpers provided by the Java implementation of handlebars by jknack

Miscellaneous

  • Due to reverse proxy nature the clients are independent of the protocol of the actual endpoint
  • Has support for forward-proxy via enable-browser-proxy

Final Thoughts

So now that we have looked into both the tools lets look at what are few advantages and limitations over each other. The list is in no way complete but can act as a quick guide to identify which tools better fits your use-case.

Hoverfly Advantages

  • Support for SPY Mode where few API's are simulated while others are passed to the real endpoint
  • Defined and Documented Simulation Schema
  • Run-time importing/resetting of the simulations
  • Supports multiple endpoint with single instance (at-least in the proxy mode)

Hoverfly Limitation

  • Cannot capture requests in webserver mode
  • Web-server mode does not support HTTPS
  • Dependent on the client for support and use of system/manual proxy settings

Wiremock Advantages

  • Independent of the protocol of the actual endpoint
  • No configuration/changes required by the client.

Wiremock Limitations

  • Runtime import/resetting of simulations not supported. Simulation mappings needs to be defined at the start during creation/initialization.
  • Cannot capture all headers. Needs a list of the headers that needs to be captured.
  • Simulation Schema though defined is not documented completely anywhere.
  • Only one endpoint can be used with one instance. (Capture needs to be specified with a target url).

July 19, 2019

HTTP API Simulation using Wiremock


In today's world of micro-services we typically integrate to many REST API (working on HTTP or HTTPS) for our application. We already saw one such simulation tool called Hoverfly.

Wiremock is another similar tool for simulation of the API's. Take a look at the documentation of the tool. It supports both standalone (or command-line) mode as well as Java API.

Lets look at a sample API simulation example using Wiremock with Java.

We will use maven to create a new application and add the following dependency
<dependency>
    <groupId>com.github.tomakehurst</groupId>
    <artifactId>wiremock-jre8</artifactId>
    <version>2.23.2</version>
</dependency>

Once the dependencies are installed we want to instantiate and start WireMock as below :
WireMockServer wireMockServer = new WireMockServer(WireMockConfiguration.wireMockConfig()
//        .dynamicPort()
//        .usingFilesUnderDirectory("~/simulation/")
          .port(8888));

wireMockServer.getOptions().filesRoot().child(WireMockApp.MAPPINGS_ROOT).createIfNecessary();
wireMockServer.getOptions().filesRoot().child(WireMockApp.FILES_ROOT).createIfNecessary();

wireMockServer.start();

Note that above before starting the server we actually create the necessary directories where the simulation files will get stored. They are stored under 'mappings' (as given by MAPPING_ROOT) and '__files' (as given by FILES_ROOT) under the root directory. The default root directory is 'src/test/resources'.

Once the server is started we want to start recording and capture the requests. This can be done as below :
// Start Recording the request and proxy all the requests to http://echo.jsontest.com
wireMockServer.startRecording("http://echo.jsontest.com/");

// Create a client to call the echo url. Note that we are calling 
// the wiremock server which will act as a proxy to the configured url
HttpClient client = HttpClientFactory.createClient();
HttpGet request = new HttpGet("https://localhost:8888/a/b/c/d");
HttpResponse response = client.execute(request);

System.out.println("Response Code : " + response.getStatusLine().getStatusCode());
BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
StringBuffer result = new StringBuffer();
String line = "";
while ((line = rd.readLine()) != null) {
    result.append(line);
}
System.out.println(result.toString());

// Stop the recording. The recorded mapping are stored inside the root directory
// and returned as result
SnapshotRecordResult recordedMappings = wireMockServer.stopRecording();

So now we have captured the recording it is time to test the simulated response. Here the recording is stopped but the wiremock server is still running. It will serve all the captured responses for similar requests.
Lets try it out
// We call the same URL and this time we will get simulated response
HttpGet same_request = new HttpGet("https://localhost:8888/a/b/c/d");
HttpResponse simulated_response = client.execute(same_request);
System.out.println("Response Code : " + simulated_response.getStatusLine().getStatusCode());
BufferedReader simBR = new BufferedReader(new InputStreamReader(simulated_response.getEntity().getContent()));
StringBuffer simBuffer = new StringBuffer();
line = "";
while ((line = simBR.readLine()) != null) {
    simBuffer.append(line);
}
System.out.println(simBuffer.toString());

// We call a different URL. As this is not captured it will return Not Found in the response
HttpGet different_request = new HttpGet("https://localhost:8888/e/f/g/h");
HttpResponse no_response = client.execute(different_request);
System.out.println("Response Code : " + no_response.getStatusLine().getStatusCode());
BufferedReader noBR = new BufferedReader(new InputStreamReader(no_response.getEntity().getContent()));
StringBuffer noBuffer = new StringBuffer();
line = "";
while ((line = noBR.readLine()) != null) {
    noBuffer.append(line);
}
System.out.println(noBuffer.toString());

At the end we need to stop the wiremock server as below
wireMockServer.stop();

Above we used record-replay to simulate the endpoints. Wiremock also provides methods for creating custom responses by using stub().

So happy simulating :)


June 29, 2019

Mocking API Requests using Hoverfly Java

 In today's world of micro-services we typically integrate to many REST API (working on HTTP or HTTPS) for our application. For development and testing we need an environment to work with those API. For this typically we write stubs or create a test-web-server to simulate those API. 

Hoverfly is one such tool that allows us to have API simulation. Take a look at the documentation of the tool in the standalone (or command-line) mode (Check out the basic tutorial). Its pretty straight forward and simple.

For development it may be feasible to use the command-line tool, but for the automated tests in the build process having this setup may be a bit of effort to setup the environment and use it. 
Instead Hoverfly also has Java (and Python) bindings for the tool that we can use in the automation.

Lets look at a sample mock API example using Java.

We will be using maven to create a new application and add the following dependency

 <dependency>
     <groupId>io.specto</groupId>
     <artifactId>hoverfly-java</artifactId>
     <version>0.12.0</version>
 </dependency>

Also make sure you are setting the Java compiler version in the properties

<properties>
    <!-- other properties, if any -->
    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.target>1.8</maven.compiler.target>
</properties>

Now lets use hoverfly to capture and store the requests :
// Initialize Hoverfly instance
try (Hoverfly hoverfly = new Hoverfly(HoverflyConfig.localConfigs().captureHeaders("Content-Type", "Date"), HoverflyMode.CAPTURE)) {
    // Start hover-fly
    hoverfly.start();
    
    // Create a client to execute http requests. OkHttpClient comes bundled with Hoverfly. You can use any http-client.
    OkHttpClient client = new OkHttpClient();
    Request dateReq = new Request.Builder().url("http://date.jsontest.com").build();
    
    try (Response response = client.newCall(dateReq).execute()) {
        System.out.println("REAL : " + response.body().string());
    } catch (IOException e) {
        e.printStackTrace();
    }
    
    // Save/Export the response to a file
    hoverfly.exportSimulation(Paths.get(System.getProperty("user.dir"), "capture_requests.json"));
}
You can check the capture_requests.json to view the request-response that got saved.
Now lets simulate the request
// Initialize Hoverfly in what they call SPY mode
try (Hoverfly hoverfly = new Hoverfly(HoverflyConfig.localConfigs(), HoverflyMode.SPY)) {
    hoverfly.start();
    
    // Initialize client to making http request
    OkHttpClient client = new OkHttpClient();
    
    // As we have not yet imported any simulation, we will get the response from the real API call
    Request timeReq = new Request.Builder().url("http://time.jsontest.com").build();
    Request dateReq = new Request.Builder().url("http://date.jsontest.com").build();

    try (Response response = client.newCall(timeReq).execute()) {
        System.out.println("REAL : " + response.body().string());
    }
    
    try (Response response = client.newCall(dateReq).execute()) {
        System.out.println("REAL : " + response.body().string());
    }
    
    // Now lets import the simulation.
    // Remember we had captured 'date.jsontest' request which will give simulated responss. Any other requests
    // will still call the real API and give the response
    hoverfly.simulate(SimulationSource.file(Paths.get(System.getProperty("user.dir"), "capture_requests.json")));
    
    try (Response response = client.newCall(timeReq).execute()) {
        System.out.println("REAL : " + response.body().string());
    }
    
    try (Response response = client.newCall(dateReq).execute()) {
        System.out.println("SIMULATED : " + response.body().string());
    }
} 
Above we used captured request simulation but you can easily build your own simulation quickly using what they call 'dsl'
So lets start Simulating :)

June 29, 2017

[Windows]: Executing a script on Windows startup

If anyone has used Linux and have been working on how to have a script executed at machine startup, that person will tell you how simple it is to achieve it.
Having written this myself for a Linux based server where the script performed startup of all required components for my project and did the necessary initialization sequence and other stuff.
Along came a client who had a Windows 2012 server and I was tasked with replicating the same as the Linux box on Windows. So I started writing a Powershell script to do the startup sequence of my project. Well, it was not as smooth and simple task (as was with Linux) to write a similar Powershell script but finally after a few brain damaging moments (pun intended) I finally managed to complete the script.

So only the final step pending was to have the script execute on machine boot. Well, this step sounded really simple until it hit me hard and after literally banging my head against the walls
I finally managed to complete it *correctly*. While there are blogs/tutorials giving a guide for the same it was really useless because I wasn't getting my script to work. After much research and some more head-bangs-against-the-wall the below steps finally worked :
  1. Press the Windows key and type in "Task Scheduler"
  2. Once the result is found right-click the entry and click "Run as administrator". If it is not started as Administrator, the script which we are about to specify will not run correctly. (This was one of the head-bang moment)
  3. Once the application opens, click "Create Task.." as shown in the image
  4. Fill in the details :
    • Give Name and Description as appropriate
    • Select the User Account with which it should run
    • Select "Run whether user is logged on or not."
    • Check Run with highest privileges.
    • Check Hidden
    • Provide "Configure for" as the current OS version which this will run on.
  5. Goto the Trigger Tab
    • Click New
    • Specify the necessary Trigger details as appropriate
      • For executing on machine reboot select "At Startup"
      • Click Ok
  6. Goto Action Tab
    • Click New
    • For the Action select "Start a program"
    • Since the Task Scheduler will run the script as a normal batch script and not a powershell script. To execute a powershell script in the Program/Script specify the path to powershell executable. 
    • This is typically "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe"
    • Add the arguments as "-executionpolicy bypass .\{{script-filename}}.ps1 > {{output-log}}.txt 2>&1"
    • Add "Start In" as the directory where the script is located
    • Now the arguments deserves some explanation :
      • Execution Policy specifies how/if should the execution begin. This is more related to using Powershell and the way Windows System handles execution. This is way too complex and you can read more on this here if you want (or go the simple route of specifying it as bypass).
      • Next is the script file name but notice we are giving relative path. You can also optionally give absolute path.
      • Next is the redirection of the output and/or we also redirect the errors to the same output
      • Note : here that it is different from the other guides as we do not specify the "-file" option. This was the head-band-multiple-times-against-the-wall moment because that is when I realized that giving the -file options (among the many other issues also due to this) is the reason why the redirection of the output and errors was not working.
    • Click Ok
  7. Click on the Conditions Tab
    • Change the settings as appropriate 
  8. Click on the Setting Tab
    • Change the settings as appropriate 
  9. Click OK
  10. And we are done :)
The output logs, error logs really helped me iron out the few issues that still were present in the script. But to get that working was a real strange mystery (and finally realizing the "-file" option as the culprit was a bummer).

Thanks and have a good day :)