A data lake becomes data swamp When it accumulates information without a catalog, owner, quality controls, lineage, permissions, or lifecycle. To avoid this, six operational controls are required: ownership, metadata, layered quality, traceability, security, and economic management. The goal is not to store more data, but to maintain reliable, findable, and usable data for analytics and AI.
Taxonomic definitions
1. Data lake
Repository designed to store large volumes of structured, semi-structured, and unstructured information, typically storing data in formats close to their original state. Its flexibility allows for integrating multiple sources for analytics, data science, and artificial intelligence.
2. Data swamp
It is a data lake that lost its governance: users no longer know for sure what it contains, where each piece of data comes from, which version is valid, who is responsible, and whether it can be used. Google Cloud explicitly identifies this risk when the data loaded into a lake is not properly cataloged, managed, and governed.
3. Data lineage
The registration of where a data originates, how it is transformed, why it passes through systems and where it ends up being used. Microsoft points out that lineage enables supporting quality analysis, compliance, fault diagnosis, and the impact of changes.
Comparative table
| Variable | Governed data lake | Data swamp | Business impact |
| Discovery | The datasets are cataloged and searchable. | The data exist, but no one knows where they are. | Hours of lost time for analysts and engineers searching for information |
| Ownership | Each asset has a responsible person | The datasets become “orphaned” | Unowned problems and longer resolution time |
| Quality | Automatic rules before consumption | Quality is revealed in the report | Rework, wrong decisions and less confidence |
| Lineage | The source, transformation and consumer are known. | The origin of a figure requires manual research. | Slow audits and changes with greater risk |
| Security | Classification and minimum privilege | Inherited permissions and excessive access | Information and regulatory risk exposure |
| Life cycle | Defined retention, filing and deletion | Everything is kept “just in case it might be useful someday” | Continued growth in storage and risk surface area |
| Economy | Cost is measured per product/use case | Only the total platform bill is known. | ROI difficult to defend before Finance |
| Consumption | There are certified datasets for business | Each area creates its own version | Different figures for the same question |
Technical team
What turns a data lake into a data swamp?
The problem rarely starts with technology. It starts with an apparently innocent premise:
“Let’s keep everything for now; we’ll see later what it’s for.”
At first it works. The storage is relatively inexpensive and adding a new source seems simple. The problem appears when the number of systems, tables, files, users and transformations increases.
Then the uncomfortable questions begin:
- Which of these five customer records is correct?
- Who is responsible for updating this dataset?
- Did this figure come from the ERP or was it modified later?
- What exactly does this field mean?
- When was it last updated?
- Does it contain personal information?
- Which dashboards would stop working if we changed the structure?
- Why do we keep paying for storage and processing for data that nobody ever looks at?
That is the point at which the data lake stops reducing friction and begins to generate it.
At MaIA by Scanda We address that problem within Data Intelligence: we do not treat the data repository as an isolated storage project, but as a capability that must be integrated. MDM, government, quality, multiple sources and analytics to produce reliable decisions.
Google Cloud sums up the technical issue well: loading information without cataloguing it, managing it, and governing it can turn a data lake into a data swamp where the data becomes difficult to find, trust, and use.
The difference, therefore, is not in how many terabytes can be stored. It is in how much of the stored data can be converted into reliable decisions.
Control 1: no data enters without owner, purpose and contract
The first defense against the data swamp occurs before storing the data.
Each new source should be entered through a minimum onboarding contract that identifies:
- business owner;
- Technical owner;
- system of origin;
- business purpose;
- the domain to which it belongs;
- expected structure or scheme;
- update frequency;
- level of sensitivity;
- expected consumers;
- retention policy;
- minimum quality standards;
- SLA or expected availability.
The operational rule
Data without an owner is not an asset: it is a future debt.
A file can physically exist for years without anyone wanting to take responsibility for its semantics, updates, or quality. When that happens, every new project must re-investigate the data from scratch.
Financial impact
Without ownership, the cost appears in the form of:
- hours of engineering to understand existing datasets;
- duplicated pipelines;
- manual reconciliations;
- delays in BI and AI projects;
- model rework;
- decisions built on different sources.
That expense rarely appears as a line called “data swamp.” It appears spread across payroll, consulting, cloud, and delayed projects. That’s why it ends up affecting the Data and AI ROI and, eventually, EBITDA.
Control 2: cataloging and metadata from ingestion
Keeping a file does not mean knowing that it exists.
A governed data lake needs a catalog that describes its information assets. At a minimum, each productive dataset should include:
- understandable name;
- business description;
- owner;
- source system;
- domination;
- scheme;
- date of last update;
- sensitivity;
- access policy;
- expected frequency;
- consumers;
- relationships with other datasets;
- lineage.
AWS uses the catalog precisely as the central piece for storing and managing metadata from a data lake and linking them with access controls.
The catalog must be operational, not just a document.
A common mistake is to build a catalog as a separate project and then ask the teams to maintain it manually.
That gets old fast.
The alternative is to capture metadata during the actual ingestion and transformation. When a new dataset appears, its schema, location, date, origin, and technical relationships must also appear automatically.
Then the human layer is added: business meaning, owner, and usage rules.
Indicator that we recommend monitoring
In our diagnostic model we also observe the number of versions of the same figure, The percentage of integrated sources and the time it takes for a data point to become a decision are very direct signals of whether the architecture is reducing or increasing friction.
If Finance, Commercial and Operations present three different values for “net sales”, the problem is not the dashboard. The problem is deeper down.
Control 3: data quality by layers, not at the end
Quality should not be validated when the CFO opens the report.
It must be verified as the data progresses through the architecture.
A widely used practice is to separate the layers according to their degree of preparation:
Bronze: keep the original data
The Bronze layer receives information practically as it arrives from the source.
It should serve to:
- preserve history;
- allow reprocessing;
- keep evidence of the original data;
- audit subsequent changes.
It should not be the official source of business indicators.
Silver: clean, validate and standardize
Here, controls must be implemented such as:
- schema validation;
- treatment of null values;
- duplication;
- normalization;
- unit standardization;
- reference rules;
- resolution of late registrations;
- Quarantine of invalid records.
Databricks recommends precisely an architecture in which the data evolves from Bronze —raw—, Silver —validated— and Gold —ready for business—, By progressively increasing its quality and reliability.
Gold: publish reliable business products
Here the data prepared for:
- executive dashboards;
- financial indicators;
- analytical models;
- applications;
- AI cases;
- Operational decisions.
The rule is simple:
Raw does not mean bad; it means not certified for commercial consumption.
The error occurs when an organization allows any application or user to access any layer without distinguishing between received, validated, and certified data.
Control 4: lineage and observability from end to end
If a field in the ERP changes, the data team should know about it before doing so:
- which pipelines depend on it;
- which tables will be changed;
- which indicators consume it;
- what AI models use that information;
- Which executive reports could be affected.
That requires lineage.
Microsoft describes lineage as the cycle that allows tracking data from its origin and understanding how it moves and transforms, using it for root cause analysis, quality, compliance, and impact.
But lineage alone is not enough
The behavior of the data must also be observed.
A serious control should monitor:
- last update;
- delay against the expected frequency;
- received volume;
- schematic changes;
- percentage of records rejected;
- duplicates;
- values outside the range;
- pipeline failures;
- quality degradation;
- datasets without consumption.
Example
Suppose the historical average daily sales is 150,000 records.
Suddenly there are 18,000.
The pipeline can technically be finished with a state “success”.
From the infrastructure point of view, everything worked.
From a business perspective, probably not.
Data observability must detect that difference. A successful process does not guarantee reliable data.
Control 5: classification and access with minimum privilege
A data swamp not only creates analytical problems; it also expands the surface area of exposure.
The organization needs to know:
- what data are public;
- which ones are internal;
- which are confidential;
- which contain personal information;
- what information requires additional protection;
- who can consult each category;
- who can modify it;
- who can export it.
AWS Lake Formation, for example, allows you to separate permissions on metadata and on the underlying data, as well as apply granular access to databases, tables, columns, rows, and cells.
The inherited permissions error
The dangerous pattern is:
“Give him access to the full bucket because he needs a table.”
Over time, dozens of exceptions, privileged accounts, and permissions appear, the original reason for which no one remembers.
The correct control is:
- minimum privilege;
- role-based or attribute-based access;
- sensitivity classification;
- periodic reviews;
- traceability of access;
- automatic revocation when the need disappears.
In our framework, preparing information for AI specifically includes cataloging sensitive information, correcting inherited permissions, reviewing privileges, and mapping sources before enabling new use cases.
Financial impact
The poor ranking increases:
- regulatory risk;
- cost of audit;
- cost of research in the face of incidents;
- potential scope of a leak;
- Time required to deploy new AI cases.
When Security does not trust the repository, each initiative takes longer to be approved. That delay also has a cost.
Control 6: Life cycle and data economy
“Storing is cheap” can be a bad government policy.
The problem isn't just how much it costs to store a terabyte. You also have to pay for:
- copies;
- supports;
- processing;
- consultations;
- movement;
- cataloguing;
- protection;
- auditing;
- maintenance;
- engineering.
That is why each category of information needs a lifecycle policy.
Define from the beginning
- How long it remains active;
- when it is transferred to less expensive storage;
- How long it should be kept;
- when it is deleted;
- which regulation requires its retention;
- which event allows it to be destroyed;
- who authorizes exceptions.
AWS S3 enables you to automate transitions to lower-cost storage classes and remove expired objects through lifecycle policies. Microsoft also recommends aligning storage levels and retention policies with actual access patterns to control costs.
Don't just measure storage
The most useful KPI is not:
“We have 300 TB.”
It is asking:
- How much does it cost to maintain each data product?
- How much does it cost to process it each month?
- How many active consumers does it have?
- When was it last consulted?
- What result does it support?
- How many copies exist?
- Which part can be archived or withdrawn?
A dataset that no one uses but is processed every night is a small recurring charge. Thousands of them are already a financial decision.
How to measure whether the data lake remains under control
At MaIA by Scanda, we believe that good governance should be measurable. Our Data Intelligence model is focused on turning governed data into reliable decisions by integrating quality, governance, MDM, and multiple sources.
An executive dashboard of the data lake should focus on indicators such as:
| KPI | What it reveals |
| % productive datasets with owner | If there is real responsibility |
| % of assets with complete mandatory metadata | If the catalog can be used |
| % from critical sources with lineage | If impact analyses can be carried out |
| Compliance rate of quality rules | If the data arrives reliable |
| Freshness versus SLA | If they arrive while they are still serving |
| Number of versions of the same metric | If there is a source of truth |
| % of datasets without consumption | How much data debt exists |
| Cost per data product | If architecture scales economically |
| Given time → decision | If the data really improves business speed |
| % of classified sensitive information | If the risk is under control |
AWS recommends setting quality thresholds, continuously evaluating them, integrating alerts, and making quality metrics visible to the consumers of each asset.
The most dangerous signal is not a bad metric.
It's not having the metric.
Predictive conclusion
The data swamp will be a bigger problem with the adoption of generative AI and corporate agents.
Before, poorly documented data could remain buried for years in a folder. Now, AI systems can find, summarize, combine, and use information at a much faster rate.
That changes the risk.
An incorrect data can feed a model. Excessive permission can expose information. A contradictory definition can lead an agent to perform an action later on.
That is why the data lake government is no longer a storage discipline to become a Trustworthy infrastructure for AI. Google Cloud, in fact, already builds its catalog capabilities around metadata, quality, lineage, and business context for analytical applications and AI agents.
At MaIA by Scanda We work under the premise that it makes no sense to accelerate AI based on a foundation of data that no one can explain. Our CoE envisages a data platform governed by quality, security, and governance as cross-functional capabilities; not as activities that are added at the end of the project.
The question for a CIO should no longer be:
“How much storage do we have available?”
The correct question is:
“What percentage of our data can be found, understood, trusted, audited and used today without manual intervention?”
That percentage will become increasingly closer to the percentage of data that can actually generate returns.
CTA final of MaIA
Your data doesn't need another repository. They need control.
If your teams are still reconciling numbers manually, unaware of which sources are official or have AI projects stalled due to quality issues, the next step is not to incorporate more technology.
At MaIA by Scanda We can evaluate the architecture, quality, catalog, governance, and flow of your data to identify what is hindering its use in analytics and artificial intelligence.
Transform your data lake into a reliable data platform for decision-making and AI, before further information accumulation increases the cost of the problem.
Schedule a Data Intelligence session with MaIA by Scanda.
FAQ
1. How do I know if my data lake has already become a data swamp?
There are five particularly clear signs: there are multiple versions of the same information, users don’t know which dataset to use, there are no owners, the analyses require manual reconciliation, and no one can easily explain the source of a figure.
A practical test consists of asking the team to find the official customer dataset, identify its owner, explain its origin, and show its latest update. If it takes several calls and messages to respond, the problem already exists.
2. Do I need to replace my data lake to fix a data swamp?
Not necessarily.
In many cases the problem is not the platform but the lack of controls. It can be corrected by incorporating cataloging, lineage, ownership, classification, quality rules, layered architecture, and lifecycle management over the existing infrastructure.
At MaIA by Scanda, we prefer to first diagnose the underlying problem. Switching platforms without correcting governance practices often transfers the same problem to a new infrastructure.
3. How is investing in data governance financially justified?
The business case must be built against observable costs:
- hours spent reconciling information;
- duplication of pipelines;
- delayed reports;
- infrastructure without consumption;
- defects caused by quality;
- AI projects detained for data;
- time dedicated to audits;
- risk associated with sensitive information;
- decisions made with inconsistent figures.
The discussion with Finance should not be about “how much it costs to govern data,” but How much it costs today not to be able to trust them.
Español 
English