Last year, we set up a prediction model on crime in London. We had established the model already, grounded in open data, but updated it to make predictions about 2017. We took the data provided by the police in the greater London area, and by enriching this data with Points Of Interest from Ordnance Survey and UK Census data, we created multiple predictive models with Dataiku in order to give these predictions for 2017 at the local LSOA level.

The model was reasonably accurate, considering we only had access to open data (taking into account the level of control we have on open data models). But let’s break down how we established our model’s performance.

Building A Data Preparation Flow for Aggregating on Monthly Observations vs Predictions

The first step was to collect the 2017 police data. I just downloaded the data (fairly) manually onto my computer. The partitioning system of Dataiku adjusts to fit the collected files’ structure, which is how Dataiku controls inserting and updating dataset rows into a meaningful organized structure. Partitioning also helps automate the recurring tasks implied by big data usage.
I partitioned the data based on the month, which helps automate our workflow. Here’s what the updated project flow looked like:

After preparing our data and joining the predictions and the LSOA boundaries, we could compare our predictions to the real observed data. To do that, I computed the residuals, numerical differences between the values as prediction – observed_crimes:

Now we could compute different indicators and reshape our data in order to analyse the predictions:

Graphically Establishing Our Model Performance

It was then time to generate some insights into the process, which are useful for analyzing the data. On a monthly basis, our R^2metric—an essential reading on model accuracy—is 0.88, which is fair especially when using limited datasets. When we look at the full year, the R^2 metric for LSOA is 0.95, which is better than expected, with a global prediction error of 8.7%.

The areas (LSOA) where the residuals are the highest are blue (when the prediction was lower than the actual) and red (when the prediction was higher) on the map. As expected, the predictive model tends to underestimate the number of crimes, which is particularly apparent in the London city centre.

To break down the model performance for 2017:

• Median Average Error (MedAE): 19 crimes
• Global fit: 95%
• Difference vs reality 8.7%

On a monthly basis, extreme residuals were observed in June, September, and December. This highlighted some limitations of the model. One way to improve it would be to add some features related to public events or weather conditions.

Let’s Automate Everything!

I created a scenario that will calculate the accuracy statistics for the previous month. Here are the steps:

1. Build the joined observed data and predictions for the previous month (or a specific month as a partition passed by the DSS API if the data are delivered later, this can be easily automated thanks to the DSS Public API
2. Refresh a Jupyter notebook containing some charts and metrics.
3. Build others datasets and refresh the charts cache for sharing the updated insights in a dashboard.

Developing the monthly automation only took a couple of hours at an airport, since Dataiku makes it really easy to push predictive projects into production. This way you don’t have to keep updating them manually, always have up-to-date projects, and most importantly, you retain control of your predictive models. Learn more about pushing analytics into production with our free white paper.

Dataiku will be presenting at Data Natives– the data-driven conference of the future, hosted in Dataconomy’s hometown of Berlin. On the 22nd & 23rd November, 110 speakers and 1,600 attendees will come together to explore the tech of tomorrow, including AI, big data, blockchain, and more. As well as two days of inspiring talks, Data Natives will also bring informative workshops, satellite events, art installations and food to our data-driven community, promising an immersive experience in the tech of tomorrow.

The pace of life and industry is accelerating at an unprecedented rate. Interconnected tech, inconceivably fast data processing capabilities and sophisticated methods of using this data all mean that we’re living in fast-forward. The Data Natives Conference 2018 will be exploring life at an accelerated pace, and what rapid innovation means for cutting-edge tech (blockchain, big data analytics, AI) across industries.

From governments to genomic projects, the quickening of life, work and research impacts every industry- and Data Natives 2018 offers two intense days of workshops, panels and talks to explore this impact. With more than 100 speakers presenting over 48 hours, the breadth of expertise at DN18 is vast; luckily, we’re here to help you curate your conference experience. The Data Natives content team have selected six talks that perfectly encapsulate this year’s topic and focus- trust us, these are six presentations you can’t afford to miss!

1. A 21st Century Paradox: Could Tech Be the Answer to Climate Change?

Climate change is one of the greatest concerns of our lifetime- and many are wondering if technology holds the answer to decelerating the impending climate disaster. Dr. Patrick Vetter of Supper und Supper will be presenting one use case which demonstrates the tangible benefits of ecotech: “Wind Turbine Segmentation in Satellite Images Using Deep Learning”. In layman’s terms, Dr. Vetter will be sharing the details of his project to optimise wind turbine placement using deep learning and analysis on “wind energy potential”. Exploring the potential of rapidly accelerating data technologies to curb the rapid acceleration of climate change, Dr. Vetter’s talk is definitely one to watch.

2. Cutting Through Propaganda: Government Policy Priorities in Practice

Any citizen of a democracy knows that there’s usually a huge gulf between the promises made in government officials’ election manifestos and what actually becomes policy. Cutting through the propaganda, is it possible to find a quantitative measure of the government’s priorities (and how they shift) over time? American Enterprise Institute Research Fellow Weifeng Zhong has been working on just such a measure: the Policy Change Index (PCI). Running machine learning algorithms on the People’s Daily, the official newspaper of the Communist Party of China, Zhong has found a way to infer significant shifts in policy direction. The PCI currently spans the past 60+ years of Chinese history- through the Great Leap Forward, the Cultural Revolution, and the economic reform program- and can now also make short-term predictions about China’s future policy directions. Zhong will be allowing us to glimpse under the hood of the PCI at Data Natives 2018, as well as sharing some of the more remarkable findings with us.

3. Blockchain: Beyond a Buzzword

Over our four editions of Data Natives, we’ve seen blockchain emerge from a promising but niche sphere into a full-blown game-changing technology. However, blockchain and decentralised computing are still shrouded in hype, and have a long way to go to garner full consumer trust. That’s where Elke Kunde, Solution Architect, Blockchain Technical Focalpoint DACH at IBM Deutschland, comes in. Her talk on “Blockchain in Practice” at Data Natives 2018 aims to demystify blockchain, slash through the hype, and enlighten the audience about how IBM clients are already using decentralised computing in their tech projects. This talk is a must-see for anyone who’s excited by the promise of blockchain, but still unclear on how exactly decentralisation can change the tech game- and their business- forever.

4. Using Machine Learning to Predict (and Hopefully Prevent) Crime

Predictive policing has been a hot topic for many years- and the technical methods behind it have become more sophisticated than ever before. Du Phan, a Data Scientist at Dataiku, will walk DN18 attendees through one particularly sophisticated model, which uses a variety of techniques including PostGIS, spatial mapping, time-series analyses, dimensionality reduction, and machine learning. As well as discussing how to visualise and model the multi-dimensional dataset, Phan will also discuss the ethical principles behind predictive policing- and what we can do to prevent crime rather than predict it.

5. Putting a Price on Personal Data

Data privacy and the price of personal data have been hot topics for years, coming to a boil with events such as the Cambridge Analytica scandal. Even Angela Merkel has declared that putting a price on personal data is “essential to ensure a fair world”- but how do we put a price on data, and how can this be enforced? Jennifer Lyn Morone- the artist who registered herself as a corporation and sold dossiers of her personal data in an art gallery- will discuss her perspective on these issues in a closing keynote for Data Natives which will bring the ethics of data science into focus.

Data Natives will take place on the 22nd and 23rd November at Kuhlhaus Berlin. For tickets and more information, please visit datanatives.io. 

The old saying goes “It’s not what you know, it’s who you know”- and in the tech industry, this sentiment is truer than ever. When careers are forged by meeting the right employer or investor at the right time, networking is a vital skill- thus, conferences become a key facet in many career paths. Some see conferences as a perk of the job, while many others see them as a necessary evil; naturally, the Data Natives team definitely form into the former camp. With decades of combined experience attending tech events all over the globe, we’ve picked up a few skills and strategies to make your conference experience as pleasurable and productive as possible.

Being the charitable souls we are, we wanted to share with you five of our favourite strategies to enhance your tech conference experience. If you have any hacks of your own, feel free to drop them in the comments!

Choose Your Conference Wisely

To say that we’re living in a boom for tech events would be an understatement. Every single day, there’s a dozen excellent tech events globally vying for your attention, and figuring out which events (and how many of them) to attend can feel like a daunting task.
It often helps to start with one clear goal in mind- from there, you can figure out what type of event is best going to suit your needs. If you’re looking for co-founders, intimate conferences with round-table discussions will give you a better depth of conversation with a good number of people. If you’re looking to rapidly expand your customer base, go big- look at your options for getting the word out at a large summit. If you’re looking for new hires, a more niche and subject-focused event is going to give you a better chance of finding an employee with the exact kind of skills you’re looking for. If you’re looking for investment, a broader tech conference is going to give you the opportunity to meet more VCs with diverse portfolios.

Figure out what your company needs most right now, and take it from there. A clear use case will also help you convince your company to send you to that conference, which is always an added bonus.

Preparation, Preparation, Preparation

This should almost go without saying, but we meet far too many people who don’t prepare before a conference. Figure out who you want to meet from the attendees & speakers lists, and make contact before the event (bonus points for getting a meeting before you step into the conference). We’ve all been to events where the networking app plays up, the matchmaking session is underattended, or the booths just aren’t inspiring- while the spontaneous connections are sometimes the best ones, the less you leave to chance, the better your chance of getting a decent ROI out of the event.

Also, use your team to help prepare. Romain Doutriaux (EMEA Marketing VP, from Data Natives 2018 partner Dataiku) makes the most out of his conference experiences by consulting the team about who to meet, and setting up meetings: “Working hand-in-hand with the customer team (SDRs+Sales) will help you to define the relevant targets you want to meet at the event,” he told us. “Emailing, social network announcements and more will let you schedule business meetings ahead of the event and therefore not rely only on luck or booth design to engage with relevant prospects.”

Solidify Existing Relationships

An often overlooked facet of the conference experience is using that time to solidify your existing contacts. There’s a lot to be said for maintaining long and meaningful professional relationships, but all too often we see companies running around, having surface interactions with dozens of contacts and flubbing the follow-up. Don’t just think about new contacts, but leveraging existing ones who are also in attendance. A simple social media call-out to see who from your network will also be going can reinvigorate old dialogues, and spark ideas for new collaborations.

Practice Your Pitch

Conferences can often feel like speed dating- and the key to taking that interaction beyond small talk between panels and into meaningful dialogue is a slick elevator pitch. Know what you want to say about yourself and your project in a few, concise sentences. The key here is being able to convey your message in a sharp and engaging manner, without sounding too sales-y – and nailing this is fantastic practice for future pitch meetings to boot.

From the initial pitch, the follow-up should cater to how the conversation progresses. “The introductory pitch should be the most solid and clearly explanatory of your business and aim. Then imagine other scenarios based on the way the relationship evolves,” says Amanda Rino, Data Natives’ Head of Operations and Projects. “The best method is to get into a conversation that is beyond pitching and something more natural. Don’t forget to find out the interests and needs of potential partners. Learn as much as you can about them and show them how a relationship with you is a no-brainer.”

Treat Conferences as a Marathon, Not a Sprint

Moreso than many other industries, tech has a reputation for attracting always-on, productivity-obsessed, 4-hours-of-sleep-a-night types- but nobody is immune to burnout. Let’s be honest: conferences can be exhausting. Several days of travel, talks, dozens of meetings and spontaneous business opportunities is exciting, but also draining.

While some people might be in their element being on their feet and engaged for 16 hours a day, for many of us, it’s important to budget time to eat, sleep, and recharge. Be smart about planning meetings, and try to avoid to many back-to-backs wherever possible; it’s way too easy to let every meeting run five minutes over, and before you know it, you’re running an hour behind. Taking some time to decompress, or just eat a good meal, is going to allow you to stay sharper and engaged for longer, and nail those vital meetings when the time comes.

We hope these tips help you up your conference game, and make the most out of the networking opportunities that come your way. If you’re joining us for Data Natives 2018 in Berlin in November and want some more guidance on how to achieve your personal goals at the event- get in touch! My Inbox is always open, and we want to make DN18 the best possible experience it can be for our whole community.

Titled ‘Data Science for Banking & Insurance’, Dataiku’s free eBook includes recommendations, use cases, testimonials, and how-to checklists that enable you to make your mark in this new era of Fintech and Insurtech. Whether you are working in marketing, risk management, product design, finance, actuarial science, underwriting, or claim management, this ebook illustrates how banking and insurance can seize the analytics opportunity. Get your free copy here.

In the internet era, giants of the digital age like Google, Apple, Facebook, and Amazon (GAFA) in Western markets and Chinese powerhouses like Baidu, Alibaba, Tencent, and Xiaomi (BATX) in Eastern markets have been increasingly straying away from their bread-and-butter products and testing the waters in large, established industries like banking. GAFA and BATX are beginning to offer services like online and mobile payments, money transfers, personal lending, account and savings management, peer-to-peer lending (crowdfunding), and currency trading.

And it’s not only the tech giants that are moving in. Countless startups in financial services have also been flooding the space and gobbling up market share, cherry picking high-volume services tailor-made for the online and mobile world into which they were born. Though large banks recovered from the global financial crisis of the early 2000s to continue to serve customers, they quickly started to lose ground as those customers turned to faster, more cutting-edge solutions to meet their financial needs.

The Rise of InsurTech

Unlike the banking industry, GAFA and BATX have not made direct forays into insurance, though the number of Iinsurtech startups in this space is on the rise. The market is ripe, as younger generations are used to ease of mobile apps and one-click shopping, and they want the same with insurance; they are not interested in the heavy process and expense associated with traditional insurance.

As in banking, peer-to-peer is hot in insurance with older players like Friendsurance and also newcomers such as Lemonade, InsPeer, InSured, and Teambrella. Each promises insurance that is more transparent and social with shared costs – things that have wide appeal in today’s market where customization is king.

Another interesting area in insurtech is item-specific, event-specific, and on-demand coverage – “smart insurance.” Startups in this space collect data about a customer’s possessions and provide machine-learning enhanced risk pricing for single-item coverage of any duration. This model allows premium levels to scale down to pennies with durations down to the second for completely customized coverage.

Insurtech and the Internet of Things

Aside from insurtech startups, the Internet of Things (IoT) is also poised to change the insurance industry in the coming months. Though IoT has been around since the 1970s, it has only recently started to infiltrate all aspects of consumers’ lives. Billions of sensors, computer processors, and communication devices are being embedded in or attached to every kind of ordinary thing imaginable, from watches to agriculture crops to cars.

And we’ve just begun to scratch the surface; Gartner estimates that by 2020, there will be more than 21 billion connected devices. Considering there were only around 3.5 billion smartphones in the world in 2015, this is astronomical growth.

Currently, the manufacturing, healthcare, retail, and security industries lead in the IoT sector, but insurance companies are well positioned to take advantage of this space. Given the upcoming ubiquity of smart homes and cars (like Nest and any number of the developing self-driving cars), a new generation of products based on real-time monitoring, collection, and analysis of data coming out of these products is on the horizon.

Ride the IoT Wave

To stay relevant in the age of IoT, some insurance companies are partnering with insurtech startups, particularly for devices in the smart home business. While GAFA is investing heavily in IoT, thus far, they have largely decided not entered the insurance market directly, leaving this space for traditional insurers to step in (for now).

And the most savvy insurance companies are beginning to step up and realize the potential of IoT. For example, insurance companies have partnered directly with device manufacturers like Water Hero, which monitors and displays water flow in real time and offers a robust alert system and remote shut-off capabilities. It’s easy to see the appeal in this partnership given that roughly one-third of all household claims are related to water leaks.

The most popular items in smart homes right now deal primarily with security and access (from light control switches and dimmers to remote security and smart doorbells), with obvious appeal to insurers. But other startups like Water Hero are creating more specific IoT devices that will certainly help prevent costly claims, particularly smart smoke and carbon monoxide detectors and mold detection. All of these devices open up the door for insurance in the time of IoT.

Staying Relevant

Of course, even when insurance companies partner with IoT manufacturers, the question still remains: who owns the customer relationship? For complete control of the customer experience and customer proximity, it’s essential that today’s insurance companies embrace the age of algorithms and better leverage IoT technology and big data to drive innovation.

Insurance can’t continue to simply partner with IoT manufacturers for long – they have to lead the movement. This means appropriating the very tools giving their new competitors an advantage in both IoT and non-IoT spheres: big data and algorithms. By leveraging IoT technology to gather more data about customers’ homes, cars, and even the people themselves, insurance companies can then better use real-time data, predictive modeling, and machine learning to create new business models and new offerings for clients.

For example, by becoming more connected to customers’ data, not only will business improve vis-a-vis claims reduction (think of the Water Hero use case and potential disasters averted with IoT), but it’s also easy to see how overall customer experience will also be significantly improved.

Customers, of course, will similarly benefit from avoiding the hassle of a claim and repairs, but on top of that, IoT can create a better experience in case there is an accident. Think of a smart car that is involved in a crash – with real-time data, the process of knowing exactly what happened and who was responsible for the crash becomes infinitely easier, more concrete, and more transparent..

Additionally, innovative developments in insurance will expand providers’ value proposition with customers. Using data and predictive data science will give more flexibility to offer customers only services and coverage that they will use. Clearly, people are looking for this offering given the number of startups in this space. But with IoT, traditional insurance companies can also compete in the space.

Algorithms are the way forward

Current startups in the space are proving that the age of algorithms is a positive development for the insurance business itself and for its customers, who are looking for more options, flexibility, and transparency, all of which IoT and big data analysis can offer.

IoT is moving forward at an astounding pace, and developments in IoT deeply entwined with and affecting insurance will continue whether insurance companies choose to get involved or not. So leveraging and investing in big data in insurance seems like an obvious win – what are you waiting for?

Like this article? Subscribe to our weekly newsletter to never miss out!

Follow @DataconomyMedia

Image: makototakeuchi, CC 2.0