Scope should include use of open standards to enable machine-to-machine interoperability with other web services and applications. This is, in my view, one of the most important aims of the work, with some of the greatest potential benefits.

The group NEEDS to address and inform the production of census data and metadata. Production should be carried out with a clear view of dissemination in order to achieve a coherent process. We're discovering that some of the 2001 outputs, particularly counts of households with particular combinations of persons (see table CAS 118, for example), are proving very difficult to fit into a hypercube model. If output production isn't informed by dissemination, we risk trying to fit square pegs into round holes all over again.

The questions as shown in the explanatory volume, also show the structure for the hierarchical codelists too.

The first things that we (mostly Rob) did in the CAIRD Project were to look at the potential XML schemas (DDI and SDMX) in order to familiarise ourselves with them and check on the kinds of information that could be encoded and the ways in which this is achieved within the schemas. Once we'd assured ourselves that the schemas could accommodate the kinds of information that we knew were entangled within the existing 2001 outputs, we set to work trying to extract the information in structured and usable forms as decribed by Rob above. The initial stages of this involved a mixture of interactive tidying up and programmatic parsing of textual table frameworks that we had already produced as an intermediate stage in the creation of the html cell selection frameworks used in our Casweb interface. Rob and Richard Wiseman created versions of the frameworks in which all the row and column headers were straightened out and filled in with values. This involved a lot of rather fiddly work, especially for some of the more complex compound tables in our sample, which had to be split into several different tables. Once the table frameworks had been restructured in this way, the codelists and their constituent codes were compared against a set of cleaned up standard codelists that Rob built up as he went along in order to make sure that all the text labels for codes were consistent (the 0-4, 0 to 4, 0-4 years, 0 to 4 years, aged 0-4 problem described by Rob previously). Rob was then able to process the table frameworks to extract the cell IDs with their associated codelist/code pairs as contained in the final table in his description.

We then assigned

Category names to each group of column and row headers

For a table of Age by General Health by Sex, the first row category would be General Health, then second row category would be Sex and the column heading category would be Age
eg

Then for each cell id we extracted the row and column headings, together with their categories and inserted this into a table.
ie

We did this for for each of tables, then we needed to harmonise the column and row headings as the syntax used varied across the tables, eg Aged 0 to 4 years old, was expressed as 0-4, 0 to 4, 0-4 years, 0 to 4 years, aged 0-4 and so on.

This then created for each category a distinct list of codes, we then used these lists as the codelists for SDMX.