The Best Way to Manage Future IT Data Growth is to Follow These Three Steps – (2/3)
This series of posts was originally going to be about the easiest data architecture and management steps to take, but no, none of these are easy. They are hard. Very hard in fact. But quite possible necessary. Let me know what you think!
The second step in our managing data growth roadmap!
(Step 1 – Future Proofing with a Storage API)
Step 2 – Manage Data by its Value
Actively, formally, automatically, intelligently assess your data’s value. Then pare it down violently. Keeping all raw data is great when exploring alien planets, but most of us won’t ever need to go back more than a few days or weeks into the full torrent for any reason. We are now data rich to the point of becoming dangerously sick from data gluttony! There will always be more data going forward and the digital world (and its inhabitants) are changing so fast that last year’s user profile data is almost already useless. Marketers want to know what the market inhabitants are up to now.
There are many large-scale streaming analytical processing solutions that can already today profile, abstract, aggregate, and reduce data streams into potentially valuable data specks, if not pebbles or even nuggets. Do everyone a favor in your organizations and lasso your incoming data streams as far upstream as possible – you’ll help boatloads just by creating consistent and authoritative baseline data dictionaries to use in aggregation that everyone downstream will now inherently share. Reducing your dirty data flood into potentially rich soup reduces storage costs, increases analytical insight, fosters collaboration, and enables new more competitive applications.
Note that I consider this step a necessary part of the annual number one IT initiative to “automate, automate, automate!”. Automation isn’t just about avoiding help desk calls or lowering staff costs (although those are benefits), it’s now got to be also about automating smart data ingestion. So steer some IT automation initiative towards data refinement and control!
(post featured image – 2023 AI generated image from Craiyon.com)