About nikhilpal

Performance, Scalability Geek, Problem Solving, Puzzles, Lego Fan. Building products

Smart ad displays

So I was standing at the Chatswood station, listening to songs and minding my own business, when my gaze fell upon one of those gigantic tv displays playing an advertisement for a random surfing channel. At the same time, Spotify with its annoying yet brilliant form of targeted advertising, started playing  a commercial about Bond university. This got me thinking, the amount of capital these companies spend for these digital billboards on the station must be enormous, yet it doesn’t have a specific target audience. I mean, if you are a company that creates an anti-aging cream, you wouldn’t want your ad to be played at a time and location where the majority of the audience is 18-25-year-olds, right?

There’s a very simple solution to this issue, though.  Imagine a display smart enough to connect with your phone and find out your preferences, likes, dislikes and displays an ad, specifically for you.

Planning on taking up running? A Nike ad plays at the mall! Looking to buy a new laptop? a Dell ad is to be seen everywhere! Interested in pop music? Get the latest Katy Perry album at 20% off!

With the amount of metadata that our phones have been collecting from us, all it takes is an AI smart enough to filter it and we have a system that can recognise the likes,  dislikes, brand loyalty and brand preference of pretty much every individual on the planet! How is that for target advertising?

Although this technology raises a few questions in the privacy and ethical departments but you have to admit that just a few years down the road, maybe two, maybe five, we’ll be looking at smart, personalised billboards that will change displays based off of your likes and dislikes, sent from your phones.



Nginx comes to rescue !!


Measure Data Usage of a Client/User in a Scalable manner @ server 

Problem Statement was very clear, we wanted to measure the data usage of users to understand the data efficiency of our product and usage patterns.

This was inherently a hard problem to solve and if you look around everyone has built custom solutions to solve this problem, especially when you need to measure at an API/Session level. As the application information is embedded in packet and the measuring is happening at an upper layer.

Fortunately in our case the User and API information is at the URI level :).

We use Netty as our application server and we tried multiple ways to measure the data usage at an API level but couldn’t do so. We tried using the SimpleChannelHandler and built some counters which would be incremented based on the MessageEvent. But due to various reasons it didn’t work, thats for another post.

Anyway as we grew as a business the importance of this metric/feature increased drastically. I re-looked at all the options and then there was a Eureka moment. What about Nginx ? Use Nginx as a pass through proxy and measure the incoming and outgoing traffic.

Experiment : was very simple, setup a Nginx instance before the Netty server as a pass through proxy and setup appropriate log_format. The one of importance are body_bytes_sent, request_length, bytes_sent. Ran a client and wireshark, measured the API calls via Wireshark and Nginx Log and they were exactly the same. Problem solved.

Next Step: Build a service which parses the Nginx log and pushes the metrics to a data source from various nodes. Make this data source accessible to the reporting module so that we can run all kinds of Analytics and make the product better.

Stats: While doing this experiment, I measured that the there is an overhead of around 300bytes on each API call, which is due to headers. Which will lead me into the next problem of optimising our API usage. 

Car remotes are so dumb


How often have you parked your car and returned to it to see whether you have locked your car or not or more importantly it’s locked or not. Sometimes kids leave the door ajar, sometimes it’s semi closed and in neither cases the remote would be able to lock the car.

Presently the feedback you get when you lock the car is either visual or audio from the car. It makes a beep or blinks the indicator to tell its locked or unlocked. For me that doesn’t work and I have been thinking of what we can do to change this.

If we have a smart remote which communicates with car, and gets a feedback from the car when the operation was successful or not and displays it to the user either via a small display or some LEDs, wouldn’t that be great.

I would be happy to pay extra to get this remote for sure 🙂

Happy and safe parking !!!

maximise your returns

GoGet is a great service especially when you don’t need car on a daily basis, I love the simple and clean process of booking. You can rent a car for minimum 2hr.

Now I wanted to rent a car for a block of 96hrs but unfortunately there was booking for 2hrs in between during the 96hrs, hence I could not do the booking. As a business GotGet lost business for 94hrs.

How can we solve this problem, what can we do to make sure we don’t loose the flexibility of allowing users to book for 2hrs but also maximise the returns.

Some assumptions:
1. The resources are different, each car is at different location.
2. Once a booking is confirmed you want to honour that.
3. All customers are equal

I have been thinking about this problem and had few ideas :
1. Keep different resources with different block times, for e.g. some resources have a block period of 2 hrs  and some resources have a block period of 24hrs.
2. Build a bidding system on top of the existing system, where you put your request for 96hrs and the existing booking can be requested to move to another resource (with some incentives, like 10% on the current booking etc.). This way we honour the booking but also get the new business. The only catch is this process wont be real time and might take 24hrs to confirm.

I am sure this is a very generic problems in lot of industries where resources are rented for e.g. Hotel Rooms etc.

Thoughts ?

electronic vault for credit cards

My wife misplaced her purse for 8hrs while I was travelling and she had to cancel all cards 😦 the good part was the she got the purse back, the bad part that she had called all the banks and cancelled the cards.

I use my credit card as direct debit option across multiple services both geeky and utility based and to be honest I don’t have a list of services where the credit cards are linked.

So for the next 4-6 weeks, I was monitoring my emails, surface mails, sms to  look for any message which talks about payment failure and then going manually and updating the service with the new card.

This whole process was painful, had periods of suspension or discontinuity, required lot of overhead.

As a customer I am happy to pay getpocket few $’s who can keep a track of all my credit card subscriptions and be a single dashboard when such an incident occurs.

Imagine the GetPocket Dashboard

– List of Credit Cards which are subscribed 
– List of services which are direct debit with due dates and amount
– Option to suspend/update the Credit Card which in turn will update the subscriptions.

Paypal does this to some extent but then PayPal is not used everywhere.

customising my dialler for smartphones

Last few days I had few messages which had phone number in them, none of them were in the format which the default dialler of the phone could understand and dial the number.

Then starts a tussle between remembering the number and typing into the dialler or using notepad where you convert the number into a proper country format and then copy the number and paste into the dialler.

Its incredible that its such a common problem and almost everyone faces it, still no one has solved it. The solution is pretty simple as well, use a library like libphone which converts a raw phone number into canonical phone number based on rules. Also the phone is aware of its MCC & MNC hence we can easily convert the raw phone numbers into canonical. Most of the apps when they ask phone number they do a similar task to normalize the phone numbers.

Once this task is done, then its easy to pass the appropriate number to the dialler which can immediately make the call without any fiddling.

Obviously there would be cases where the normal rules would not fit and you still would have to make changes to phone number but that should be rare.

The biggest challenge is to get something done like this on iOS which is a closed ecosphere, you cant change the dialler or do some smart things while you open the dialler app. On the other hand Android should be pretty straight forward.

Hopefully after reading this blog a smart engineer with some spare time, develops this utility 🙂


e-commerce sites want do lucky draw


Every business wants to attract new customers and a very common way of doing it is by doing a marketing campaign, a survey where everyone participates and the winner is chosen via a lucky draw.

For small businesses its quite a task to do this, they have to outsource it to a third party vendor.

This website/service is going to reduce the friction for any business to allow to do this.

You register an account, provide all the user-ids & names which need to go into the lucky draw. You set the time when the lucky draw is set to happen. The service will provide with a unique URL which when clicked will show the details about the draw.

At the time of draw it would use random.org and generate a random number and then declare the result.

The admin can declare the result based on the lucky-draw service.

Its a very simple tool and will be really useful.


Following the rule of three, I have had at-least three instances where I would love to use a service like this. The service is a very simple service just like IFTTT but for AWS. It will provide triggers on which you can start/stop the AWS instances or do specific task on AWS instance.

Here are the three use cases for which I wanted a service like this :

1. Running a Jenkins box, only when you need to trigger a build, you could have a github trigger with some time cap. For example build only once an hour, and queue the request for next hour.

2. When you are starting up a project and working part time, you don’t need the instances up & running 24×7, you could have a email trigger, start/stop the app-server

3. Run an instance at a fixed time,  and do some specific task, in my case download few feeds/videos and push it to s3 public website.

priceline gobbles up kayak

Travel industry has a super heavyweight, priceline.com has acquired kayak.com for whopping $1.8B. http://news.smh.com.au/breaking-news-technology/priceline-negotiates-18-billion-kayak-deal-20121109-2923c.html 

I was reading through the quarterly results of kayak.com and found useful data http://ir.kayak.com/releasedetail.cfm?ReleaseID=719891

Some of the key things for the 3rd quarter

  • 246 million queries on Big Web
  • 56 million queries on mobile
  • Big Web Revenue Per Thousand  Queries $305
  • Mobile Revenue Per Thousand  Queries $65
  • Revenue from non-US geographies was $17.3 million compared to $61.3 from US

For every 1 query on mobile there are around 5 queries happening on web, and similarly for every $1 revenue on mobile there is 5 times more revenue on big web. This kind of tells if we increase the reach of mobile than we will get equivalent revenue.

Great work Kayak team, kudos for giving us a great tool to plan our travel…


real estate search

We haves been searching a property to rent long enough that I have spent multiple hours using domain.com.au and realestate.com.au sites. I have used iPhone & iPad apps and the big web to persue the task at hand.

They are great sites no doubts about it but I think that’s because of the content and not due to actual presentation. The data is exclusive to these sites, as a developer I don’t have to access to this data. This closed information stifles the innovation which could happen in this area.

I have at least three ideas which would make this site more useful and give a better perspective on renting or buying.

Here are the ideas in priority order, advantages, business value

  • Map mash up : iPad app already draws the property on google maps, my idea is to enhance this further have more layers on top of this layer of 131500 bus routes ability to draw a polygon in maps and restrict the search in that polygon
    Once you have the public transport you can show the travel times easily
  • Ability to hide some properties : as you don’t like them, it’s annoying to have them in your search results even though you don’t want them
  • can we share anonymous data of previous data like previous tenants, children, single, married etc
  • panoramic picture of the house, is that really hard. Given the new apps in the market it’s pretty easy job to stitch all of them together and make a perfect 360 view.
  • publish stats or trends more often, these guys are sitting on a treasure, it’s such rich data, can we publish any information we can gather from the data. It will make the consumer informed, add traction to the site.

These features are pretty easy to implement, none of them require huge re-architecture.

Would love to hear from other users or the product owners of the above sites and what they have to say about the feature list.