MicroArray Gene Expression (MAGE) Loader for BioWarehouse

Version 4.6

(C) 2006 SRI International. All Rights Reserved.  See BioWarehouse Overview for license details.

Introduction
Installation and Building
MAGE Schema Generation
Schema Creation Algorithm
Loader Limitations
Documentation & References

Introduction

This document describes the MAGE Loader. The loader loads microarray gene expression data in the form of MAGE-ML into the BioWarehouse. MAGE and MAGE-ML are standards for gene expression data defined by the Microarray Gene Expression Data Society (MGED). The MAGE standard is defined by an Object Model (OM). The MAGE schema extension of the BioWarehouse is an algorithmic transformation of the OM into a relational database schema, described below.

Input data for the MAGE loader must be in the form of MAGE-ML (an XML file). MAGE-ML data is available from a variety of sources. (For example, the European Bioinformatics Institute's ArrayExpress public repository for microarray data at http://www.ebi.ac.uk/arrayexpress). The loader will accept data from any source, provided the data is in the MAGE-ML format.

Installation and Building

Please see the MAGE Loader installation instructions for details on installing, building, and running the loader.

MAGE Schema Generation

The MAGE BioWarehouse DB schema is generated from the MAGE Object Model (OM). The schema generation algorithm follows the one table per class hierarchy approach to object-relational mapping. (See link for discussion of object-relational mapping strategies.) Each top-level MAGE class (those classes which are at the root of an inheritance hierarchy, or those than directly extend the MAGE classes Extendable, Describable, and Identifiable) corresponds to one relational database table of the same name. Classes that extend these base classes have their properties added as additional columns in the table. Each table has a discriminator column ("MAGEClass") that indicates the specific name of the class that row in the table represents. See also the the cross-referenced MAGE-to-BioWarehouse mapping documentation.

For example, Parameterizable is a base class that extends Identifiable. Three classes inherit from Parameterizable: Hardware, Protocol, and Software. The table representing the Parameterizable class hierarchy includes the following columns representing the four classes' attributes and associations to other MAGE classes:

(This is an abbreviated table description. See schema documentation for full description of this table.)

Table Parameterizable

Column	Description
WID	Warehouse identifier
MAGEClass	MAGE class that this row in the table represents
name	Represents MAGE class attribute: Identifiable.name
URI	Represents MAGE class attribute: Parameterizable.URI
model	Represents MAGE class attribute: Hardware.model
make	Represents MAGE class attribute: Hardware.make
text	Represents MAGE class attribute: Protocol.text
title	Represents MAGE class attribute: Protocol.title
Protocol_type	Represents MAGE Association: Type{3B72CBE201C8} This association is a 1..1 association between OntologyEntry and Protocol [type] OntologyEntry 0..1 [] Protocol 1..1 The type of a Protocol, a user should provide/use a recommended vocabulary. Examples of types include: RNA extraction, array washing, etc...
hardware	Represents MAGE Association: Software2Hardware{3B041BB50345} This association is a 1..n association between Hardware and Software [hardware] Hardware 0..1 [softwares] Software 0..n Associates Hardware and Software together.
Software_type	Represents MAGE Association: Type{3B72CBE600B5} This association is a 1..1 association between OntologyEntry and Software [type] OntologyEntry 0..1 [] Software 1..1 The type of a piece of Software. Examples include: feature extractor...
Hardware_type	Represents MAGE Association: Type{3B72CBDF0259} This association is a 1..1 association between OntologyEntry and Hardware [type] OntologyEntry 0..1 [] Hardware 1..1 The type of a piece of Hardware. Examples include: scanner, wash station...
DataSetWID	DataSet.WID this entity is associated with

Schema creation algorithm

For each root MAGE class:

Create table with root MAGE class name
If this table represents an inheritance hierarchy, add a "MAGEClass" discriminator column, which will indicate the name of the specific subclass a row represents.
Add all attributes and associations of root class to table
Add all attributes and associations of any ancestor classes (Extendable, Identifiable, Describable)
Recursively add all attributes and associations of any child classes
Add a WID column as the primary key of the table
Add a DataSetWID column with a foreign key reference to DataSet.WID

Association handling. For these examples, assume the relationship is between class A and class B represented by tables X and Y, respectively.

1..1 associations

Add a column to table X with a foreign key reference to the WID column of table Y.
The name of the column is given by the name of class A, followed by an underscore, followed by the name of the association. If this combination produces an identifier name that is longer than the maximum allowed by the Oracle or MySQL RDBMS, then the name of the association alone becomes the name of the column.
If the association is bidirectional, also implement the reverse association by adding a column to table Y referencing the WID of column X.
For example: see Parameterizable.Protocol_type in the example above.

1..n associations

Add a column to table Y (the n end of the association) with a foreign key reference to the WID column of table X (the 1 end of the association).
The name of the column is given by the name of the 1 end of the association. If the 1 end of the association does not have a name, the name of class A is the name of the column. If this combination produces an identifier name that is longer than the maximum allowed by the Oracle or MySQL RDBMS, then the name of the association alone becomes the name of the column.
For example: see Parameterizable.hardware in the example above.

n..n associations

An associative linking table is created. The name of the table is "<name of class A>WID<name of class B>WID". In some cases, the table name produced by this convention is longer than the maximum allowed identifier length. In that case, the table names are shortened. For example, "ReporterDimensionWIDReporterWID" becomes "ReporterDimensWIDReporterWID".
Two columns are defined for this table, "<name of class A>WID" with a foreign key reference to the WID column of table X and "<name of class B>WID" with a foreign key reference to the WID column of table Y.
If the association is recursive, that is if the linking table represents an association between rows of the same class and table, then the columns are named "<class name>WID1" and "<class name>WID2".

Identifier naming conventions

Identifiers that exceed the maximum allowed length are shortened as described above.
Identifiers that are reserved words have an underscore added to the end of the identifier. (E.g. table "Audit" becomes "Audit_")
In the case where the property or association mappings produce columns of identical name, the second a subsequence column name will have a numeral appended to the end of the column name.

In some cases, MAGE concepts overlapped with existing BioWarehouse constructs. Tables that were redundant with BioWarehouse tables were not generated, and MAGE classes corresponding to those concepts were mapped to the appropriate BioWarehouse table. See the MAGE-to-BioWarehouse mapping documentation for details on specific MAGE classes.

Loader Limitations

Only limited support for SeqFeatures is provided. All SeqFeatures currently map to the BioWarehouse Feature table as the MAGE OM provides no unambiguous way of determining a SeqFeature type. SeqFeatures with a single SeqFeatureLocation will populate Feature.StartPosition and Feature.EndPosition. Other SeqFeature geometries will be reported by the loader as warnings and not stored.
Some portions of the MAGE OM are not commonly used, and examples of their usage are hard to come by. Therefore, there may be some types of MAGE-ML that we have not tested against. If you encounter any problems in loading your MAGE-ML, please contact us at support@biowarehouse.org.
Properties not loaded by the MAGE loader are marked as <not handled> in the MAGE-to-BioWarehouse mapping documentation

Documentation & References

The MAGE standard is defined by the Microarray Gene Expression Data Society (MGED), includeing the MAGE Object Model and the MAGE-ML data format.
MAGE OM in XMI (UML) format (doc/MAGE.xmi)
MAGE-ML DTD (doc/MAGE-ML.dtd)
ArrayExpress has a very useful browseable MAGE OM.
MAGE-to-BioWarehouse mapping documentation