Inconsistent fico line item data for updating
Redundantly kept data -- that is, data derived from other data available elsewhere in the database -- is one of the big challenges of software systems.You may wonder why you would store, for example, the sum of two invoice amounts separately if the same value can easily be derived by calculating it on the fly from the two invoices.In the past, such data redundancy was only introduced to increase performance, because traditional databases could not keep up with user expectations in light of billions of data entries.This came at the costs of significant effort to keep the redundant data consistent, increased database storage and more complex systems.This chapter first introduces the conceptual benefits of a redundancy-free system by contrasting it with the disadvantages of redundant data (Section 3.1).Section 3.2 demonstrates how SAP HANA and in-memory technology enable us to overcome redundancy and avoid its pitfalls.A materialized aggregate does not contain one-to-one duplicates, but nevertheless, the data is redundant, because it can be derived from the original data at any time by applying the same calculation on the fly.Duplicated data due to overlap Data may be duplicated if related information is stored in separate locations.
Materialized result set Long-running programs that arrive at a certain output after several steps of calculation often store the result of their work as a materialized result set.
In contrast to redundancy within the same database, this cross-system duplication may in some cases be wanted -- for example, for data security purposes.
In other cases, it is truly redundant -- for example, in the case of data warehouses, due to missing analytical capabilities of traditional database systems.
Redundant data is distinguished by the fact that you need to maintain its consistency.
If you modify the data at one location, then you need to apply corresponding modifications at all the other locations in the database in order to keep the relationship intact and avoid anomalies.This chapter covers the removal of redundancy with regard to the first two categories: materialized views and materialized aggregates.