AboshStart a manufacturing analytics trial with a decision your team struggles to make today. Use the sixteen criteria in this guide to check data connections, metric definitions, analysis and day-to-day operation. Then compare the six software categories against those needs. Ask who will maintain the definitions and what the complete trial will cost. Vrolen makes software in one of these categories; consider that interest when using this guide.
Sixteen criteria in four groups
Six vendor categories and when each is appropriate
A disclosed vendor interest and criteria you can test in a trial
Record six tests, owners, agreed limits, actual evidence and verdicts.
CSVFictional duplicate-event testAn expected result with actual result and verdict left blank. No vendor evaluation claimed.
Start with one question the tool must answer
Choose a recurring production question before comparing features. Which stop should the team investigate? Why did scrap rise on one product? Can the next order fit in the available capacity?
Name the person who makes that decision and the data they need. A trial should show whether the tool helps them reach a defensible answer with less manual work. The acceptance sheet below gives you a small test to start with.
Why this page has no “top ten”
A useful shortlist needs a documented evaluation, not a ranking invented from feature pages. Vrolen is also a vendor, currently offered through a private pilot and waitlist. Use the same acceptance checks for us and for other candidates.
Sixteen criteria in four groups
Score each candidate against the decisions you actually need to make. Not every criterion matters equally, and weights should reflect your plant.
| Group | Criteria |
|---|---|
| Connect | OT and machine connectivity; ERP, MES, QMS and CMMS context; event and time alignment |
| Trust | Metric-definition governance; data ownership and export; human approval and auditability |
| Decide | Descriptive analytics; diagnostic and root-cause support; predictive analytics; simulation or scenario capability; AI and agentic workflow |
| Operate | Role-based use; edge and offline needs; deployment model; time to value; total cost of ownership |
Six vendor categories and when each fits
Products can cover several categories. Check the functions included in the proposed configuration, then test them against the same operating question.
| Category | Fits when | Watch for |
|---|---|---|
| BI platforms configured for manufacturing | You have clean data and analysts who can build the views | Shop-floor context and event data need heavy modeling |
| MES and MOM analytics | Execution data is already in the MES and the questions are about it | Limited view of data outside the MES |
| OEE and production monitoring | You need machine-state visibility and OEE quickly | Strong on measuring loss, lighter on causes and scenarios |
| Industrial data platforms | You need to unify many sources at scale | A foundation, not an answer to a decision |
| Decision-centric improvement platforms | The problem is deciding what to change, and testing it first | Newer category; check maturity and how assumptions are shown |
| Digital twin and simulation platforms | You must test changes to flow, schedules or capacity | Model build and upkeep effort. See twin vs simulation vs process mining |
Ask for evidence, not demos
Run each shortlisted product against the same real decision, with your data. The manufacturing data analytics guide covers what the data must look like, and the manufacturing software guide shows where analytics sits beside ERP, MES and quality systems. For where a new tool ends and another begins, read MES vs OEE vs analytics.
- Pick one decision
For example, which recurring stop to investigate, or which schedule to release.
- Set the success measure
What outcome would show the decision improved?
- Give every candidate the same data
A representative slice, including messy periods.
- Check the reasoning
Can you see how it reached its answer and what it assumed?
- Ask about exit
Can you export your data and definitions if you leave?
Set acceptance checks before the vendor trial
Choose one question, such as “Which recurring stop should we investigate next?” Give every candidate the same approved, minimized dataset and the same definitions. Decide which checks are essential before seeing the demo. The editable sheet records the owner, expected result, evidence and pass, fail or not-tested outcome.
For data quality, include a known missing interval, a duplicate event and a late-arriving correction. Ask the tool to keep those cases visible. For timestamps, trace a source event to the displayed record. OPC UA defines a source timestamp separately from a server timestamp; a screen that only shows when data arrived can put events in the wrong order.
For a fictional arithmetic check, supply one event lasting 120 seconds and a duplicate with the same event ID. The accepted total is 120 seconds, not 240. Ask the vendor to show the deduplication rule and export both the raw records and the calculation result. Passing this small check does not establish full production readiness.
The limits in the sheet are examples to agree with your team, not industry standards. A process owner approves the operational answer, IT reviews access and export, and the data owner checks definitions. A feature you do not test stays “not tested”; it does not earn a pass because it appeared in a slide.
| Check | Test input or task | Evidence required |
|---|---|---|
| Event accuracy | 120-second event plus identical duplicate | 120-second result and visible rule |
| Time meaning | Known source and arrival timestamps | Separate fields, timezone and trace |
| Data gaps | Known missing interval | Unknown interval displayed, not filled as running |
| Drill-down | One stop selected from the summary | Source IDs, reasons and corrections retained |
| Operational answer | Same recurring-stop question | Reasoning, limits and responsible reviewer |
| Access and exit | Authorized read-only user and export | Permission check and usable export |
Common mistakes when buying analytics software
Use your own records in the trial and ask who will maintain each connection, definition and export.
- Scoring features instead of decisions.
- Judging the demo dataset instead of your own messy data.
- Ignoring who will maintain metric definitions after go-live.
- Underestimating integration effort with ERP, MES and machine data.
- Forgetting the exit: export of your data and definitions.
A related example from Abolfazl
Compare configuration, customisation and maintenance beyond the purchase price.
In English, on founder Abolfazl’s personal blog. This is related reading, not an independent technical source.
Practical checklist
- Name the decisions the software must help you make.
- Weight the sixteen criteria for your plant.
- Identify the category each candidate belongs to.
- Test every candidate on the same real decision and data.
- Check that metric definitions are governed and visible.
- Confirm human approval and an audit trail for AI outputs.
- Ask what happens to your data and definitions on exit.
- Estimate time to value and total cost, including integration.
FAQ
FAQ
Sources and further reading
Authoritative references used to research and verify this guide.