Entity Framework - How to manage suburb and state date across multiple databases - entity-framework

Question

I have an SaaS application in the pipeworks.
One of the things that has me a bit confused is the best way to manage the stable of Austalian suburb and state data across multiple databases (this applies to any country as each country has a list like this).
For example in Australia you have Australian Postcode list that links all the postcodes to the suburbs and you can use that to create a dropdown for state, suburb and postcode etc.
An example of the CSV of australian postcodes can be found HERE.
So you can upload a csv file for example but the problem remains..
Whats the best way to hold this data.. its common to all databases where you have a person, client, employee etc..
Do you replcate it in each database? Is there a better way than having redundant stores of data..
Best way to implement it..

Answer 1

There are several options and considerations I would look at for this problem. Some considerations:
Number of address rows expected
Whether a client database is concerned with prefill/validated international addresses
Whether the client system is web connected or can operate in isolation
Are these databases/systems hosted by you or distributed to individual clients? (SaaS implies "Web" and "Hosted by You" to points 3 & 4)
How critical address integrity is.
For smaller systems, a simple option for address systems is to de-normalize the address data (state, postcode, suburb) and consider using a central lookup database/service, either under your own control or a third party. The denormalized address table would contain the text fields for the State, Postcode, Suburb etc. rather than FK values (stateId, suburbId, etc.) This avoids needing to store lookup tables in every client DB, just one Lookup DB or leave that to a 3rd party service.
The advantage of a third-party lookup is that keeping it up to date with new areas and changes is handled for you. Third party services would require a web connection, and you have to factor in the risk of their service being down or a web connection being unavailable. Larger systems with millions of addresses might benefit from normalizing the address table, so the "cost" of replicating suitable address lookup tables might be worthwhile. You can still a central service to look up addresses, then resolve whether the client DB already has a StateId, SuburbId etc. for the respective state/suburb for that post-code before inserting one if necessary. (Cutting down the number of rows each client DB needs to address values that are actually used)
In that last example you might have lookup tables for State and Suburb linked to PostCodes, linked to Country. Country would default to the target, maybe be an optional selection for international addresses. The user provides a post code to the service which returns suburbs, they select a suburb. The address validation service could go as far as to validate the street address. When you're happy an address is "valid" and ready to be saved, you search your local State, Suburb, (even Street) tables for matches for that PostCode, if found use those FKs, otherwise insert new entries and link the FK.
Using a separate service, or services would be my consideration especially if you need to support validating/storing international addresses. For instance if the client is in Australia but regularly has address information for New Zealand. Storing entire address validation tables could get rather large if clients could be resolving addresses for many countries. (I.e. European countries and neighbours) You can write a Façade service to support different 3rd party address validation providers and/or homemade implementations with a standard interface.
If a system has to operate in isolation of an internet connection then you'll probably be stuck with each database having one or more local data sources to resolve address information.
Data integrity of address information is a separate concern you might want to consider. In some systems you need to validate that an address is recognized and don't want to allow invalid combinations or detect unexpected changes. Services that validate a particular address can provide unique IDs for an address that you can store as part of your address information. (These often tie into geocoordinate solutions where you want to quickly direct a map service to a particular location) Alternatively, if you successfully look up an address then validate that the address information is valid, even if just the country, post code, and suburb, you can create and store a hash of those values to check for tampering. (I.e. someone or some system changed a field to make the address invalid, the combined address won't match the stored hash) Addresses can be checked before use and flagged if not valid.