“To Excel” or “Not to Excel” – That Is the Process Engineering Question
Excel isn’t dead, but it can’t do it all.
Excel is probably the most popular spreadsheet software in the world. Regardless of your industry or profession, it’s a staple and will likely continue to remain a constant in the business world. Personally, I’m a fan. I love making Pivot tables and gettin’ that data organized. But I also know that it really can’t do everything I need it to do, especially when dealing with massive data sets, information from various applications, or trying to troubleshoot.
Within industrial manufacturing, dealing with massive data sets and process issues is part of the job, and it can take up a good portion of an engineer’s daily workload. For many of you, your toolbox go-to workflows, especially for solving challenges, involve using trending clients and spreadsheets.
Trending clients are actually not bad for data exploration, manual monitoring, and especially visual pattern recognition. For example, how many of you have tried to eyeball data looking for the trends to understand what’s going on in your process? (Of course, this also requires a heck of a lot of patience.) Trending clients are also pretty good at simple calculations.
Excel still has many benefits as well, and there will always be a need for robust spreadsheets. When it comes to analytics however, Excel comes up short in many areas (like searching and troubleshooting, for example). How many of you have spent countless hours or even days scrolling back in time examining an Excel spreadsheet trying to recognize a pattern in the data to figure out what happened? There’s also no ability to contextualize operational data or provide early warnings for abnormal behavior.
If you happen to still be using spreadsheets to analyze your data, it might be time to reassess your toolbox and consider a self-service analytics solution that will make your job – and your life – much easier.
Why “To Excel”
While a lot of the discussion surrounding new technology centers around the theme “Ditch the Spreadsheets”, Excel isn’t going anywhere anytime soon. You can accomplish a lot in spreadsheets, and many process experts still use it daily for a number of reasons:
- It’s an easy and highly available application. Practically everyone knows how to use Excel (to some extent).
- It’s flexible with a vast number of built-in functions.
- It’s good for performing quick/ad Hoc calculations, as long as it is used with a reduced data set.
- It’s also good for preparing additional graphs and visualizations when used in combination with exported data
These same benefits, however, are some of the reasons Excel continues to be used for the wrong purpose – like data analysis.
Why “Not to Excel”
There is an over-reliance on spreadsheets for aggregating critical plant information. Understandable. Yes, Excel has its place in your software landscape, but there are some important limitations which means using it alone is not enough. Especially when dealing with industrial manufacturing data.
Here are some reasons spreadsheets alone are not enough when it comes to Big Data Analytics.
It’s not ideal for operational and time-series data and time-series analytics.
Looking for patterns in immensely huge data files is just incredibly difficult and time-consuming. Additionally, you have a variety of data sources all containing critical information that remains unintegrated and messy. It becomes cumbersome causing major headaches.
Massive file sizes can create massive headaches.
A single spreadsheet has a limited amount of data it can easily handle. A spreadsheet with about 100 or even 200 megabytes of data, which is not a whole lot, can bring you trouble. Spreadsheets can become very slow and clunky, and once you finally have your huge datasets, not to mention the calculations on top, things can get worse. This can lead to data loss from crashes or workstation or laptop hang-ups which can cause big delays.
Ain’t nobody got time for that.
Next is the human error factor.
One of the biggest reasons spreadsheets shouldn’t be relied on solely for important operational decisions is the aspect of human error. A few years back, a Harvard study discovered that 88% of spreadsheets contain human errors. That means that only one in five spreadsheets will give you truly correct results.
If you want to incredulously laugh at how simple, man-made spreadsheet errors have cost companies serious money and resources, check these Excel spreadsheet horror stories.
There’s the lack of version control too.
Version Control is another limitation. Even with the advent of cloud tooling or network storage which makes this less of an issue, colleagues will send a certain version over email. You then have to rely on somebody else to put it back on the network drive, so that sharing a version is not ideal.
Another frustration is that engineers often create different versions of the same file, so data gets lost. A huge bummer and a huge potential problem of inaccurate information.
It’s also not suited for real-time analytics.
Spreadsheets are also not particularly suited for real-time analytics. If you want to reduce the time between a process event and taking action on that real time aspect, Excel doesn’t do a good enough job. A common example is when you have to shoot out an alert. To do this, you have to have Excel open to see if the device is going to work or not. It’s clumsy and impractical.
And Excel’s limited sharing capabilities means lack of context.
If you use Excel for data analytics, you’ve learned sharing your work is not easy. But for fun let’s say you have a great share drive and file collaboration isn’t an issue. What is an issue is not having context behind what your colleagues were doing, why they were doing it, and what their findings may have been. Results may remain hidden and inaccessible to all of your team.
If, for example, a process expert opens a new spreadsheet to pull in data, access to all the analyses that maybe you or someone else completed before will not be available – the context of previous analyses will not be there. So, in essence, your process experts are working in the dark, repeating work, and wasting your time and theirs.
Ain’t nobody got time for that, either.
The Alternative to Excel for Operational Data
Besides all of the “Why Not to Excel’s” that were covered above, using spreadsheets is time-consuming which means it is costly. After all, time is money, so being able to reduce the time between a process event occurring on the factory floor and taking decisions is key to saving money. It is therefore extremely important to have the infrastructure in place to identify certain events and to take immediate data-driven action.
Additionally, it is common for a kind of friction to exist in the process industry between being completely data-driven and still relying on the experience and gut feelings of your most experienced operators and engineers. What you want is to break down this barrier and find a common ground to get the best of both worlds making your decisions data-driven and backed by your experts’ process experience and instinct.
Self-service analytics lets you have this, and more importantly, lets you fully leverage your process data and the expertise and experience of your process experts.
Self-Service Analytics: Simpler, Easier, & Better
Self-service analytics empowers your whole team because they can do the analytics. Your engineers and operators will be able to analyze, monitor, and predict the processes to run your plants more efficiently. (In case you’re not sure what self-service analytics is, here’s a 2-minute video explaining it).
Self-service analytics adds value on top of real-time data infrastructure. It allows you to:
- Make time-series data ready for analytics through plug & play indexing of all historical data.
- Contextualize time-series data by adding context information as discrete or sequential events, visualize these on diagrams with search, comment, and collaborative events.
- Use fingerprints, conditions, and business rules for setting early warnings and soft alarms to capture extra events and send out notifications via text messages or emails.
- Stay aligned with existing workflows and processes by integrating alerts with 3rd party tools.
- Introduce asset awareness by using an asset structure directly.
- Use any tag for analytics, e.g., pattern searches, diagnoses, comparisons, and monitoring.
Ease Your Daily Work Challenges
Spreadsheets have their uses, but there is a much better and simpler way to do your job. Self-service analytics will ease your daily work challenges allowing you to truly get a handle on operations.
Now, you no longer have to contemplate whether “To Excel” or “Not To Excel”.