Feeds:
Posts
Comments

Archive for the ‘Decision’ Category

Mike Shipluski

 

End

 

Source: shipluski.com

 

 

Advertisements

Read Full Post »

sub-buzz-13932-1492644193-1

 

End

 

 

Source: Buzzfeed.com

 

Read Full Post »

And no maths, no equations.

This is from an article that appeared sometime ago edited for readability, deleting (irritating, light-weight and thoroughly avoidable) references to some IT specific code snippets and operations. Here you go:

How the Circle Line rogue train was caught with data

The MRT Circle Line (London underground) was hit by a spate of mysterious disruptions in recent months, causing much confusion and distress to thousands of commuters.

Like most of my colleagues, I take a train on the Circle Line to my office at one-north every morning. So on November 5, when my team was given the chance to investigate the cause, I volunteered without hesitation.

From prior investigations by train operator SMRT and the Land Transport Authority (LTA), we already knew that the incidents were caused by some form of signal interference, which led to loss of signals in some trains. The signal loss would trigger the emergency brake safety feature in those trains and cause them to stop randomly along the tracks.

But the incidents — which first happened in August — seemed to occur at random, making it difficult for the investigation team to pinpoint the exact cause.

We were given a dataset compiled by SMRT that contained the following information:

  • Date and time of each incident
  • Location of incident
  • ID of train involved
  • Direction of train

We started by cleaning the data…

This gave us:

picture-1Screenshot 1: Output from initial processing

No clear answers from initial visualisations

We could not find any obvious answers in our initial exploratory analysis, as seen in the following charts:

  1. The incidents were spread throughout a day, and the number of incidents across the day mirrored peak and off-peak travel times.

picture-2Figure 1: Number of occurrences mirror peak and off-peak travel times.

  1. The incidents happened at various locations on the Circle Line, with slightly more occurrences on the west side.

picture-3Figure 2: The cause of the interference did not seem to be location-based.

  1. The signal interferences did not affect just one or two trains, but many of the trains on the Circle Line. “PV” is short for “Passenger Vehicle”.

picture-4Figure 3: 60 different trains were hit by signal interference.

 

The Marey Chart: Visualising time, location and direction

Our next step was to incorporate multiple dimensions into the exploratory analysis.

We were inspired by the Marey Chart, which was featured in Edward Tufte’s vaunted 1983 classic The Visual Display of Quantitative Information. More recently, it was used by Mike Barry and Brian Card for their extensive visualisation project on the Boston subway system:

In this chart, the vertical axis represents time — chronologically from top to bottom — while the horizontal axis represents stations along a train line. The diagonal lines represent train movement.

Under normal circumstances, a train that runs between HarbourFront and Dhoby Ghaut would move in a line similar to this, with each one-way trip taking just over an hour:

picture-5Figure 5: Stylised representation of train movement on Circle Line

Our intention was to plot the incidents — which are points instead of lines — on this chart.

Preparing the data for visualisation

With the data processed, we were able to create a scatterplot of all the emergency braking incidents. Each dot here represents an incident. Once again, we were unable to spot any clear pattern of incidents.

picture-6Figure 6: Signal interference incidents represented as a scatterplot

Next, we added train direction to the chart by representing each incident as a triangle pointing to the left or right, instead of dots:

picture-7Figure 7: Direction is represented by arrows and colour.

It looked fairly random, but when we zoomed into the chart, a pattern seemed to surface:

picture-8Figure 8: Incidents between 6am and 10am

If you read the chart carefully, you would notice that the breakdowns seem to happen in sequence. When a train got hit by interference, another train behind moving in the same direction got hit soon after.

What we’d established was that there seemed to be a pattern over time and location: Incidents were happening one after another, in the opposite direction of the previous incident. It seemed almost like there was a “trail of destruction”…

Could the cause of the interference be a train — in the opposite track?

picture-9Figure 9: Could it be a train moving in the opposite direction?

We decided to test this “rogue train” hypothesis.

We knew that the travel time between stations along the Circle Line ranges between two and four minutes. This means we could group all emergency braking incidents together if they occur up to four minutes apart.

We found all incident pairs that satisfied this condition: We then grouped all related pairs of incidents into larger sets…This allowed us to group incidents that could be linked to the same “rogue train”…These were some of the clusters that we identified:

[{0, 1},
{2, 4},
{5, 6, 7},
{8, 9},
{18, 19, 20},
{21, 22, 24, 26, 27},
{28, 29, 30, 31, 32, 33, 34},
{42, 44, 45},
{47, 48},
{51, 52, 53, 56}]

Next, we calculated the percentage of the incidents that could be explained by our clustering algorithm. The result was:

(189, 259, 0.7297297297297297)

What it means: Of the 259 emergency braking incidents in our dataset, 189 cases — or 73% of them — could be explained by the “rogue train” hypothesis. We felt we were on the right track.

We coloured the incident chart based on the clustering results. Triangles with the same colour are in the same cluster.

picture-10Figure 10: Incidents clustered by our algorithm

How many rogue trains are there?

As we showed in Figure 5, each end-to-end trip on the Circle Line takes about 1 hour. We drew best-fit lines through the incidents plots and the lines closely matched that of Figure 5. This strongly implied that there was only one “rogue train”.

picture-12Figure 11: Time of clustered incidents strongly implies that the interference could be linked a single train

We also observed that the unidentified “rogue train” itself did not seem to encounter any signalling issues, as it did not appear on our scatter plots.

Convinced that we had a good case, we decided to investigate further.

Catching the rogue train

After sundown, we went to Kim Chuan Depot to identify the “rogue train”. We could not inspect the detailed train logs that day because SMRT needed more time to extract the data. So we decided to identify the train the old school way — by reviewing video records of trains arriving at and leaving each station at the times of the incidents.

At 3am, the team had found the prime suspect: PV46, a train that has been in service since 2015.

Testing the hypothesis

On November 6 (Sunday), LTA and SMRT tested if PV46 was the source of the problem by running the train during off-peak hours. We were right — PV46 indeed caused a loss of communications between nearby trains and activated the emergency brakes on those trains. No such incident happened before PV46 was put into service on that day.

On November 7 (Monday), my team processed the historical location data of PV46 and concluded that more than 95% of all incidents from August to November could be explained by our hypothesis. The remaining incidents were likely due to signal loss that happen occasionally under normal conditions.

The pattern was especially clear on certain days, like September 1. You can easily see that interference incidents happened during or around the time belts when PV46 was in service.

picture-13LTA and SMRT eventually published a joint press release on November 11 to share the findings with the public.

Final thoughts

When we first started, my colleagues and I were hoping to find patterns that may be of interest to the cross-agency investigation team, which included many officers at LTA, SMRT and DSTA. The tidy incident logs provided by SMRT and LTA were instrumental in getting us off to a good start, as minimal cleaning up was required before we could import and analyse the data. We were also gratified by the effective follow-up investigations by LTA and DSTA that confirmed the hardware problems on PV46.

From the data science perspective, we were lucky that incidents happened so close to one another. That allowed us to identify both the problem and the culprit in such a short time. If the incidents were more isolated, the zigzag pattern would have been less apparent, and it would have taken us more time — and data — to solve the mystery.

Of course, we were most pleased that all of us can now take the Circle Line to work with confidence again.

Daniel Sim, Lee Shangqian and Clarence Ng are data scientists at GovTech’s Data Science Division.

 

End

Source: https://blog.data.gov.sg/how-we-caught-the-circle-line-rogue-train-with-data-79405c86ab6a

Read Full Post »

At least to me, it’s new. Never thought the joke could be on us, not about someone from south-of-boondocks as I had imagined.

A policeman sees a drunk staring at the ground beneath a streetlight. “What are you doing?” the cop asks.

“Looking for my keys.” says the drunk. “I dropped them in the dark alley over there.”

“Then why are you over here?” asks the policeman, confused.

“Because the light’s so much better over here.”

The streetlights are our controlled environments where we look for answers —labs, classrooms, fixed timetables, and clear metrics. But things are more fluid in the real world. For that we need to rely more on tacit knowledge from our experience

End

 

 

Source: conversationagent.com/2016/07/striving-for-conciseness-and-clarity.html while talking about ‘Streetlights and Shadows: Searching for the Keys to Adaptive Decision Making’, a book by research psychologist Gary Klein, a pioneer in naturalistic decision making.

 

 

Read Full Post »

Whenever there’s a line of cars backing up on the road, my guess is usually right – it’s a cop managing a traffic junction ahead instead of the traffic lights or the timers of the traffic lights set badly to treat light traffic gets just the same as heavy. The cop’s no-thanks contribution comes from his long persistence with a flow before he turns on the next.

Here’s a lightly edited excerpt from an article by Simon Jenkins appearing in The Guardian of 4th Feb 16. A case for dismantling authority! Well, that was stretching the point a little too far, admittedly. An interesting experiment in social behavior that runs counter-intuitive for many of us, nevertheless reportedly successful in outcomes. Here we go:

 

The removal of road markings is to be celebrated.

We are safer without them

White line in the middle

Sensational news. The government is starting to remove white lines from the middle of roads in parts of the UK. It is doing so to reduce accidents and save lives…

Research has shown that removing white lines induces uncertainty and thus cuts vehicle speeds by 13%. This has been the case on London’s A22, A23 and A100. Pilot schemes are also in place in Wiltshire, Derby, and round the Queen’s house at Sandringham.

Behind this demarking lies the concept of “shared space” and “naked streets”, developed in the 1990s by the late Dutch engineer, Hans Monderman. He held that traffic was safest when road users were “self-policing” and streets were cleared of controlling clutter. His innovations, now adopted in some 400 towns across Europe, have led to dramatic falls in accidents.

Monderman’s principle is that freedom to assess risk for ourselves is what makes us safer. Rules, controls, signs, traffic lights all reduce our awareness of our surroundings and thus our sense of danger. On roads, he said: “When you don’t exactly know who has right of way, you tend to seek eye contact with other road users. You automatically reduce your speed … and take greater care.”

Monderman’s British apostle, John Adams of University College, London…has figures to prove that countries… that do not require motorbike helmets have fewer biking accidents than those that do.

The most dangerous place on a road is at traffic lights. Almost half the accidents in most cities occur at lights. This is due not just to war between “amber gamblers” and reckless cyclists. A crossing is where everyone should be watching everyone else, but everyone is watching the lights. They are awaiting orders. When given them, they assume all is safe and crash on.

Traffic engineers, who maim and kill us with regulations, lights and paint pots, merely go on dreaming up ever more.

Last October the Yorkshire town of Beverley suffered a failure of 42 traffic lights and was amazed when traffic moved more smoothly. In Somerset the traffic campaigner Martin Cassini secured a 50% improvement in traffic flow by persuading Portishead to turn off its traffic lights. They stayed off.

The same goes for…the one-way street and the cycle lane. A recent US study found that making streets two-way halved accidents, shortened journeys and reduced pollution…a 2014 study suggested that painting roads made cycling more dangerous.

Monderman…saw a symptom of a deeper ailment. “We’re losing our capacity for socially responsible behaviour. The greater the number of prescriptions, the more people’s sense of personal responsibility dwindles,” he said. It was this loss of responsibility that yielded the paradox that ever more intrusive safety and security actually make us less safe. Traffic is the most visible manifestation of that.

While it is difficult to argue against data, it seems to fly in the face of the premise: systems, processes and technology take the hassles and risks out of our lives.  Don’t we need all the protection possible from that one nut who screws it up inadvertently, irresponsibly or irrationally?

 

End

 

 

Source: theguardian.com/commentisfree/2016/feb/04/removal-road-markings-safer-fewer-accidents-drivers

 

Read Full Post »

James Lather in his blog carries this interesting story about an internal memo from Howard Schultz on ‘The Commoditization of the Starbucks Experience’. Though dated, is very much relevant even today. Read on:

From: Howard Schultz
Sent: Wednesday, February 14, 2007 10:39 AM Pacific Standard Time
To: Jim Donald
Cc: Anne Saunders; Dave Pace; Dorothy Kim; Gerry Lopez; Jim Alling; Ken Lombard; Martin Coles; Michael Casey; Michelle Gass; Paula Boggs; Sandra Taylor

Subject: The Commoditization of the Starbucks Experience

As you prepare for the FY 08 strategic planning process, I want to share some of my thoughts with you.

Over the past ten years, in order to achieve the growth, development, and scale necessary to go from less than 1,000 stores to 13,000 stores and beyond, we have had to make a series of decisions that, in retrospect, have lead to the watering down of the Starbucks experience, and, what some might call the commoditization of our brand.

298485028_79fa325d78_z-e1444251366989

Many of these decisions were probably right at the time, and on their own merit would not have created the dilution of the experience; but in this case, the sum is much greater and, unfortunately, much more damaging than the individual pieces. For example, when we went to automatic espresso machines, we solved a major problem in terms of speed of service and efficiency. At the same time, we overlooked the fact that we would remove much of the romance and theatre that was in play with the use of the La Marzocca machines. This specific decision became even more damaging when the height of the machines, which are now in thousands of stores, blocked the visual sight line the customer previously had to watch the drink being made, and for the intimate experience with the barista. This, coupled with the need for fresh roasted coffee in every North America city and every international market, moved us toward the decision and the need for flavor locked packaging. Again, the right decision at the right time, and once again I believe we overlooked the cause and the affect of flavor lock in our stores. We achieved fresh roasted bagged coffee, but at what cost? The loss of aroma — perhaps the most powerful non-verbal signal we had in our stores; the loss of our people scooping fresh coffee from the bins and grinding it fresh in front of the customer, and once again stripping the store of tradition and our heritage? Then we moved to store design. Clearly we have had to streamline store design to gain efficiencies of scale and to make sure we had the ROI on sales to investment ratios that would satisfy the financial side of our business. However, one of the results has been stores that no longer have the soul of the past and reflect a chain of stores vs. the warm feeling of a neighborhood store. Some people even call our stores sterile, cookie cutter, no longer reflecting the passion our partners feel about our coffee. In fact, I am not sure people today even know we are roasting coffee. You certainly can’t get the message from being in our stores. The merchandise, more art than science, is far removed from being the merchant that I believe we can be and certainly at a minimum should support the foundation of our coffee heritage. Some stores don’t have coffee grinders, French presses from Bodum, or even coffee filters.

Now that I have provided you with a list of some of the underlying issues that I believe we need to solve, let me say at the outset that we have all been part of these decisions. I take full responsibility myself, but we desperately need to look into the mirror and realize it’s time to get back to the core and make the changes necessary to evoke the heritage, the tradition, and the passion that we all have for the true Starbucks experience. While the current state of affairs for the most part is self induced, that has lead to competitors of all kinds, small and large coffee companies, fast food operators, and mom and pops, to position themselves in a way that creates awareness, trial and loyalty of people who previously have been Starbucks customers. This must be eradicated.

I have said for 20 years that our success is not an entitlement and now it’s proving to be a reality. Let’s be smarter about how we are spending our time, money and resources. Let’s get back to the core. Push for innovation and do the things necessary to once again differentiate Starbucks from all others. We source and buy the highest quality coffee. We have built the most trusted brand in coffee in the world, and we have an enormous responsibility to both the people who have come before us and the 150,000 partners and their families who are relying on our stewardship.

Finally, I would like to acknowledge all that you do for Starbucks. Without your passion and commitment, we would not be where we are today…

James observes this memo is interesting for two reasons:

1.It shows the problem with ill-conceived process improvement that fixed an internal metric and did not benefit the customer.

2.It is all about management integrity. The CEO is a brave man who, when he sees a mistake, takes full responsibility for it and does a U-turn to solve the problem.

My two cents: A process improvement that fixes an internal metric is not such a sin as made out to be. In fact it is sorely needed with the back-office processes. But when it intersects in any way with user’s experience – the memo provides with many concrete examples in this case – one needs to look at it with much greater care and caution. The points of intersection may not be obvious at the first glance.

End

Source: ft.com/cms/s/0/dc 5099ac-c391-11db-9047-000b5df10621.html#axzz3o8YnDFyr and James Lather’s blog at squawkpoint.com. Image is from flickr.com.

Read Full Post »

This anecdote comes again from B:

Arnoud999-Right-or-wrong-


1967 March/April. I was appearing for my Intermediate examination (13th year) at a college in south Mumbai. Those days it was 11 years in school, 2 years of pre-degree in college and another 2 years in a bachelor’s degree course.

It was the day of Hindi exam. As always I did my namaskars to Lord Ganesha (the deity thought to remove all obstacles) and to my mother before leaving the house. A practice widely followed in my generation.

Most of candidates taking the examination were already employed in paying job,now wanting to earn a university degree to further their career prospects. In preparation for the same they would attend coaching classes in the morning while attending to their jobs during the day.

So here I was in the examination hall staring blankly at the question paper in Hindi. Most of us – I and others in the hall – were migrants from south, poor in Hindi, disinterested as well beyond the need of the situation. There was no choice before us as Hindi was mandated and we had to pass.

The supervisor in the hall knew the predicament. He closed his eyes to the candidates freely copying from whatever sources including text-books. And I was staring at the question paper like I was decoding script from the Indus Valley tablets. Before long the roving eyes of the supervisor settled on a-completely-unengaged me. He moved to station himself near my desk and sent out subtle signals for me to “do what you want”,

It was not difficult at all for me to decide – I wrote what I could and came away. To me, a standing injunction against less-than-honest means from my mother, a-not-so-preachy financially weak and uneducated single parent, was simply non-negotiable come what may.

When the results were declared  I was surprised to note from newspapers that I had passed!

The  mark-sheet subsequently issued showed I had scored a bountiful 6 out of 75 marks against a minimum passing requirement of 15. So the newspapers were wrong.It had to be. Though none of it was unexpected I was still devastated.

In sheer disgust I gazed at the mark-sheet hoping for some miracle. And there it was!

There was a special note at the bottom of the nothing-to-speak-of mark-sheet:  Failure in Hindi condoned under section….” Reason: I had passed / scored well on all other papers. The powers that be were taking a kinder view of the matter in their efforts to promote Hindi (as national language).

While we all conduct our lives adopting a certain code of ethics learnt either from elders in the family or in school, we differ in our intensity of compliance.

Oftentimes we cite exigencies:

 ‘We have to move with time,

We have to be flexible and realistic,

Just this once

to permit ourselves infractions that are minor to begin with, insidiously building up into major breaches over time. Every shred starts with a tear.

In the above anecdote B steadfastly holds onto his tenet of not resorting to less-than-honest means to achieve an end as simply inviolable and non-negotiable even in the face of adverse fallout’s. From there it was no longer a difficult decision for him to make.

A courage we wish we had more of.

In an earlier post (see https://tskraghu.wordpress.com/2015/06/29/a-wrong-was-righted-and-how/) B was introduced as my neighbor for years. And also the father of the highly successful and talented Vidya Balan of Bollywood.

End

.

.

.

Source: openclipart (amroud999)

Read Full Post »

Older Posts »