The Pareto principle, also known as the 80-20 rule, asserts that 80% of consequences come from 20% of causes, rendering the remainder way less impactful.
Those working with data may have heard a different rendition of the 80-20 rule: A data scientist spends 80% of their time at work cleaning up messy data as opposed to doing actual analysis or generating insights. Imagine a 30-minute drive expanded to two-and-a-half hours by traffic jams, and you’ll get the picture.
While most data scientists spend over 20% of their time on
real analytics, they still have countless hours to spend turning a lot of messy
data into a neat dataset ready for analysis. This process can include removing
duplicate data, ensuring that all records are in the correct format, and other
preparatory work.
According to a recent Anaconda poll, this workflow step
takes about 45% of the total time on average. An earlier CrowdFlower poll gave
a 60% rating, and many other reviews cite numbers in that range.
None of this means that data preparation is not important.
"Trash in, trash out" is a well-known rule in computer science
circles that also applies to data science. At best, the script will simply
return an error warning that it cannot calculate the average cost per customer
because the input for customer # 1527 is formatted as text, not a number. In
the worst case, the company will act on the basis of knowledge that has little
to do with reality.
The real question to be asked here is whether reformatting the data for client # 1527 is really the best way to use the time of a well-paid expert. It is estimated that the average data scientist makes between $ 95,000 and $ 120,000 a year. Focusing an employee on such a paycheck on mind-numbing, non-specialized tasks is a waste of their time and company money. In addition, real data has a lifespan, and if a dataset for a time-sensitive project takes too long to collect and process, it may become out of date before any analysis can be done.
Moreover, companies' searches for data often involve wasting the time of non-data-driven staff when employees are asked to help retrieve or provide data in lieu of performing their normal duties. More than half of the data collected by companies is often not used at all, suggesting that the time of all those involved in the collection was wasted in only generating operational delay and associated losses.
On the other hand, the data collected is often only used by a designated data analyst team, who are too overwhelmed with work to study everything that is available.
All for data and data for all
All of the issues outlined here play a role in the fact that, with the exception of data pioneers like Google and Facebook, companies are still racking their brains over how to reimagine themselves for a data-driven era. Data is being sucked into huge databases, leaving data scientists with a lot of cleanup work, while others whose time has been spent helping to retrieve the data do not benefit from it too often.
In truth, we are still in the early stages when it comes to data conversion. The success of tech giants that put data at the heart of their business models has sparked a spark that is just starting to take off. While the results are mixed at this point, it's a sign that companies have yet to master the data mindset.
Data is of great value and companies are very well aware of it, as evidenced by the appetite for AI experts in non-tech companies. Companies just have to do it right, and one of the key challenges in this regard is to focus on people as much as we do on AI.
Data can improve the performance of virtually any component of the organizational structure of any business. As tempting as it may be to think of a future that will use a machine learning model for every business process, we don't need to go that far right now. The task of any company that wants to get data today is to get it from point A to point B. Point A is part of the workflow where data is collected, and point B is the person who needs this data to make decisions. ... ...
It is important to note that point B does not have to be a data scientist. This could be a manager trying to determine the optimal workflow scheme, an engineer looking for flaws in a manufacturing process, or a user interface designer performing A / B testing on a specific function. All these people must always have the necessary data at hand, ready for processing and analysis.
People can excel with data as well as models, especially if a company invests in it and remembers to equip them with basic analysis skills. In this approach, accessibility should be the name of the game.
Skeptics might argue that big data is nothing more than an overused corporate buzzword, but advanced analytics can improve any company's bottom line if they have a clear plan and expectations. The first step is to focus on making the data accessible and easy to use, rather than extracting as much data as possible.
In other words, a comprehensive data culture is as important to the enterprise as the data infrastructure.
globalmarketingguide bloomersweb techbizcenter marketing2business
upticktechnology
