“The biggest issue for governments today is how to be relevant. If all citizens are treated with dignity and invited to collaborate, it can be easier for administrations to have a direct finger on the pulse of the nation rather than lose it in transmission through multiple layers of bureaucracy”.

This week has seen several landmark announcements at the intersection between big data and bureaucracy. Firstly, we discovered that the first Prime Minister to use big data to secure a win, Narendra Modi, continues to innovate in his position of power. His office is using sophisticated big data analytics techniques on the mygov.in portal to ascertain public opinion in key areas of policy, in a move which has been described as a blueprint for future democracy.

Whilst India move towards big data adoption, the EU continue to be wary. A landmark ruling in the European Parliament this week voted to “break up” Google, and separate its search functions from its other revenue streams. Whilst the European Parliament lack the autonomy to implement this ruling, it sends a strong and clear message to Google that the EU remain disdainful of their business practices. The UN also approved “The Right to Privacy in the Digital Age”, keeping data privacy and ethics firmly on the agenda.

Here’s our pick of the rest of the key data science announcements you may have missed this week.

TOP DATACONOMY ARTICLES

The Data Science Skills Network

As a data scientist, I am usually heads down in numbers, patterns, and code, but as crazy as it sounds, one of the hardest parts of my job is actually describing what I do. There are plenty of resources that offer descriptions and guides on the career of a data scientist. I’ve heard them described as those at the intersection of statistics, hacking abilities, and domain expertise. Or, as data analysts who live in San Francisco.

Why Organizations Need a Data Strategy

One of the most important tasks that a Data Architect is often asked to help with is the creation of an Enterprise Data Strategy. But why is Data Strategy so important and what exactly does it consist of, and lastly why is this a task that a Data Architect should be leading or supporting?                                                                                                                       …   

Top 5 Data Science Incubators and Accelerators

Unfortunately, one great idea does not make a successful company. When it comes to founding a data science startup, a visionary idea is essential, but so is hiring the right team, gaining an in-depth knowledge of your market, and putting in place the best business practices to help your idea flourish. And then of course there’s funding.

TOP DATACONOMY NEWS

Medium Rolls Out Slick Visualisation Tool Charted, No Strings Attached                                                                                                

Publishing platform Medium has open-sourced their proprietary data visualisation tool, Charted, on Monday. Now available for public use, data stored on Dropbox or Google Drive can be linked to charted.co and it returns a visualization of that data made shareable through a permalink.                                                                                                    …                                                                                                                                               …

Moogsoft Unveil Deep Monitoring of Docker and OpenStack on Incident.MOOG Platform

Moogsoft are the self-proclaimed “leading provider of operational intelligence for the new era of software”. To ensure that they’re firmly at the cutting edge of new technology, they’ve enhanced their Incident.MOOG software product – a service which provides deep monitoring support for hype-laden Docker container environments, as well as open source cloud computing platform OpenStack.

Qubole Provides Self-managing Data Platform as a Service on Microsoft Azure

Qubole, a self-service Platform for Big Data Analytics, announced a strategic relationship with Microsoft Azure in a step to make Big Data solutions more accessible to more people on more platforms. Big Data is an elastic workload and the ideal cloud use case. The Qubole Data Service is a great way to drive cloud services revenue while delivering improved time to value Microsoft Azure customers.

TOP UPCOMING EVENTS

2 December, 2014- Machine Learning for Sensory Data Analysis, Australia     

Following the inaugural event successfully held in conjunction with AI’2013 in Dunedin, MLSDA’14 joins PRICAI’14 to provide a forum that aims at bringing researchers on sensory technology, networking, and distributed data mining together in order to respond to the unprecedented research challenges and opportunities resulted from the synergy between big data, machine learning, and wireless sensor networking.

2-3 December, 2014- The Chief Data Officer Summit, New York

Data is the most competitive tool that your organization has in order to maintain relevance and growth in today’s complex environment. The ground-breaking Chief Data Officer Summit covers the latest innovations you need to advance your organization’s data strategy and management. In addition to the wealth of insight into data development, data governance and innovation, the summit will dissect the role of the Chief Data Officer like never before.

TOP DATACONOMY JOBS

Pricing Manager / Analyst, Wayfair   

As an Analyst of Pricing you will be responsible for pricing every product that appears on the website. You will manage the daily operational pricing functions while continually seeking to optimize procedures and test strategies to increase gross profit. If you love diving into deep data sets to identify areas for improvement, and be even more enthusiastic about solving those problems then do not hesitate to apply!

Business Intelligence Analyst, CupoNation

We are currently looking for new talent to join our highly professional and dynamic Business Intelligence team. In your role you will be responsible to design end-to-end solutions that meet our company’s Business Intelligence requirements. This covers the definition and implementation of technical requirements for ETL jobs, creation of new data layers and the optimization and enhancement of the current data warehouse infrastructure.

Moogsoft are the self-proclaimed “leading provider of operational intelligence for the new era of software”. To ensure that they’re firmly at the cutting edge of new technology, they’ve enhanced their Incident.MOOG software product – a service which provides deep monitoring support for hype-laden Docker container environments, as well as open source cloud computing platform OpenStack.

The idea of containerization – running your distributed applications within a single layer of software which can run seamlessly across all kinds of computing environments – is the latest twist on virtualisation, and one which has gained a lot of traction given Docker’s $65m funding this year alone. But containerisation in production poses new DevOps challenges; namely, if a problem occurs, it’s difficult to ascertain if it’s due to the application, the container, the cloud, or elsewhere in the IT stack. This is where Moogsoft come in – its software acts as an early warning system to detect problems as they unfold, then clusters related alerts using machine learning to help pinpoint root cause by monitoring status data pushed by the entire environment in real-time.

“About a dozen of our online and traditional enterprise customers are building Docker farms on top of OpenStack clouds right now,” said Rob Markovich, chief marketing officer at Moogsoft. “They drove the requirements for this new capability because they have no tools to give them visibility and monitoring to keep the Docker environment running smoothly.”

Incident.MOOG also allows for:

• Taking events from all areas in the container stack out of the box, including from the containers themselves, the output logs of the containerized apps, the management software like Shipyard and Ansible, and the monitoring systems like Nagios, Sensu and New Relic
• Integrating tightly into the Docker API and Openstack API to instrument the core platforms of containerization
• Providing a standard REST-based API to facilitate data integrations into Incident.MOOG (made even easier with a Node.js plug-in)
• Automatically finding anomalies as they unfold, then clustering related events according to Docker groups (linked and unlinked), Openstack cell or host UUID, and output from deployment automation platforms, e.g. Ansible, Chef, SaltStack
• Communicating over Docker API to dynamically monitor containers as they are turned up and turned down

Integrating support for innovative technologies such as Docker and OpenStack demonstrates MoogSoft’s commitment to making the lives of DevOps teams easier across ever-evolving technology stacks. If they continue to keep apace as new technologies emerge, the future certainly looks bright for MoogSoft.

Follow @DataconomyMedia

(Image credit: Moogsoft)