The two sites' annotations are prepared by the EcoliWiki team (mostly [[User:DanielRenfro|Daniel,) and submitted via CVS to the GO Consortium. The figure at right shows the four basic steps to this process:
Obtain annotations from EcoCyc (quarterly)
Obtain annotations from EcoliWiki (monthly)
Merge them and resolve name-conflicts
Submit to the GO
Detailed Description
There are actually two files that get submitted each month. The gene association file and the gp2protein file. A detailed description of each can be found in the GO Consortium's documentation.
Step 0.5 - Setup the environment
The process requires some files and scripts from the GO Consortium. Instead of do this by hand each month, Daniel wrote a script to handle this part of the process. To use this bash script, copy this code into a file and make it executable. On our servers it is named gaf_setup.sh.
The first step is to run this script. Alternatively, you can follow the directions below to do this by hand.
EcoCyc's quarterly releases contain their annotations in the gene_association file format. The file is typically named gene_associations.ecocyc.... (Notice the plural on associations). In order to prepare their annotations, we must first parse the annotations for validity.
To change this information, just edit the file from the CVS checkout and resubmit it. It will get prepended to the gene_assocation file automatically.
This script will print the erroneous lines with short descriptions of the problem. Some of these errors will be easy to fix and will not have any biological significance. A common error is misspellings or disallowed entries in the Assigned_by column (column 15.) Other errors are non-trivial and should be kept track of for further curation.
I need to make sure this information gets back to EcoCyc. --Daniel 22:43, 8 March 2010 (UTC)
Get the good lines from the EcoCyc release, for later inclusion into the GAF.
Just like with the EcoCyc annotations, we must validate the annotations. The lack of restrictions in the annotation table makes for some interesting errors, most of which can be fixed by hand. These errors need to be fixed in the wiki as well, or they will show up again next month.
Using a text editor, the errors are fixed. Some sample errors are:
a.) unprintable character found on this line, may not be this column
b.) Line identical to previous line
c.) Qualifier column=4 allowed type not present, found "Deprecated"
d.) DB:Reference column=6 format of reference not DB:REFID
Some characters, such as 'β', cannot be included as there is no standard encoding for the GAF and they can become garbled. Also, EcoliWiki does not enforce strict reference notation, so some lines may need a PubMed number.
EcoliWiki uses some non-standard qualifiers such as deprecated and under_review. These are internal qualifiers meant to tag annotations for later curation. To get a list of these, run the following command (requires ack).
The names of the products to which the annotations are made sometimes do not match. This needs manual curation and is the most time consuming part of the GAF file preparation.
First, some background:
To facilitate inter-website information exchange EcoliWiki uses the EcoCyc internal naming scheme to refer to the annotations.
This is the internal EcoCyc name for that product. EcoliWiki thus puts EG10240-MONOMER in column 2 in its gene_associations.ecoliwiki file. This is to make merging easier.
Column 2 in the GAF spec is db_object_id -- the internal name used.
Column 3 in the GAF spec is db_object_symbol -- this can be thought of as the common-name of the product, in this case: DnaJ.
Problems arise when EcoliWiki and EcoCyc use different db_object_symbols/common-names for the same db_object_id. (Column 2 matches, but column 3 doesn't.) The GO filter script counts this as an error.
To successfully merge the two websites' files, we first concatenate them together.
Using the name_conflicts.txt file, we can find all the occurrences where naming differences resulted in errors. Below is a table listing some of the common name-conflicts encountered. Where there was a discrepancy between EcoliWiki and EcoCyc, the name was looked up in EcoGene and in the literature.
Make a full list of things I've come across and send to Markus at EcoCyc. --Daniel 23:39, 8 March 2010 (UTC)
DB_object_ID
EcoCyc id
EcoliWiki id
EcoGene id
what I did in this case...
LACTALDDEHYDROG-MONOMER
Ald
AldA
AldA
changed Ald to AldA
G8209-MONOMER
MolR_3
molR_3 (molR)
yehH'
MolR_3
TRANSKETOI-MONOMER
Tkt
TktA
TktA
changed to TktA
THREDEHYDCAT-MONOMER
Tdc
TdcB
TdcB
TdcB
G7743-MONOMER
Hsp15
HslR
HslR
HslR
That's an L, not a 1
G7744-MONOMER
Hsp33
HslO
HslO
HslO
EG10600-MONOMER
GroE
GroES
---
GroES
based on what is in the description and literature
The last editing step before the submissions is to remove the IEA annotations that are older than 1 year.
gp2protein.ecocyc
Step 4 - Submit
header information
Occasionally the header information in the gene_association file needs to be updated. This information can be found in the gene_assocation.ecocyc.conf file in the go/gene-associations/submission/ directory of the GO CVS.
submission via CVS
The files () are sorted and uniqed using the appropriate Unix commands. They are then gzipped and moved into the CVS directory.
