OBITools issueshttps://git.metabarcoding.org/obitools/obitools/-/issues2023-04-08T19:26:35Zhttps://git.metabarcoding.org/obitools/obitools/-/issues/48Trouble with ecoPCR/ecotag2023-04-08T19:26:35ZMatthew HopkenTrouble with ecoPCR/ecotagHello,
After following your instructions on the wolf diet webpage, I keep having issues with generating a database from the EMBL sequences.
I can get obiconvert to process the files but it seems it is not generating the .pdx file. This seems to cause problems for taxon assignment. Here are the commands and error. Any help would be appreciated.
ecotag -d embl_last -R db_v05_r117.fasta VM061719-001.ali.assigned.uniq.c10.l80.fasta > VM061719-001.ali.assigned.uniq.c10.l80.tag.fasta Reading binary taxonomy database...
[INFO : Taxon alias file found]
[INFO : Local taxon file not found]
Taxonomical tree read
[Errno 2] No such file or directory: 'embl_last.pdx'
[INFO : Preferred taxon name file not found]
ok
Reading reference DB ... [Errno 2] No such file or directory: 'db_v05_r117.fasta'
(base) matt_hopken@abdoserver1:~/Vector_metagenomics/PR_bloodmeal$ obigrep -d embl_last --require-rank=species --require-rank=genus --require-rank=family 12sV5.ecopcr > 12sV5_clean.fasta
Reading binary taxonomy database...
[INFO : Taxon alias file found]
[INFO : Local taxon file not found]
Taxonomical tree read
[Errno 2] No such file or directory: 'embl_last.pdx'
[INFO : Preferred taxon name file not found]
ok
12sV5.ecopcr 100.0 % |##################################################/] remain : 00:00:00
Thank you!Hello,
After following your instructions on the wolf diet webpage, I keep having issues with generating a database from the EMBL sequences.
I can get obiconvert to process the files but it seems it is not generating the .pdx file. This seems to cause problems for taxon assignment. Here are the commands and error. Any help would be appreciated.
ecotag -d embl_last -R db_v05_r117.fasta VM061719-001.ali.assigned.uniq.c10.l80.fasta > VM061719-001.ali.assigned.uniq.c10.l80.tag.fasta Reading binary taxonomy database...
[INFO : Taxon alias file found]
[INFO : Local taxon file not found]
Taxonomical tree read
[Errno 2] No such file or directory: 'embl_last.pdx'
[INFO : Preferred taxon name file not found]
ok
Reading reference DB ... [Errno 2] No such file or directory: 'db_v05_r117.fasta'
(base) matt_hopken@abdoserver1:~/Vector_metagenomics/PR_bloodmeal$ obigrep -d embl_last --require-rank=species --require-rank=genus --require-rank=family 12sV5.ecopcr > 12sV5_clean.fasta
Reading binary taxonomy database...
[INFO : Taxon alias file found]
[INFO : Local taxon file not found]
Taxonomical tree read
[Errno 2] No such file or directory: 'embl_last.pdx'
[INFO : Preferred taxon name file not found]
ok
12sV5.ecopcr 100.0 % |##################################################/] remain : 00:00:00
Thank you!https://git.metabarcoding.org/obitools/obitools/-/issues/61obiconvert issue2021-02-10T09:39:05ZGabriele Cananziobiconvert issueHi everyone,
I am having this issue with obiconvert, hope someone could help me!
**obiconvert --embl -t ./TAXO --ecopcrdb-output=embl_last ./EMBL/*.dat**
Reading taxonomy dump file...
List all taxonomy rank...
Traceback (most recent call last):
File "/usr/bin/obiconvert", line 43, in <module>
writer = sequenceWriterGenerator(options)
File "/usr/lib/python3/dist-packages/obitools/format/options.py", line 364, in sequenceWriterGenerator
writer=EcoPCRDBSequenceWriter(options)
File "/usr/lib/python3/dist-packages/obitools/ecopcr/sequence.py", line 72, in __init__
self._taxonomy= loadTaxonomyDatabase(options)
File "/usr/lib/python3/dist-packages/obitools/ecopcr/options.py", line 79, in loadTaxonomyDatabase
taxonomy = TaxonomyDump(options.taxdump)
File "/usr/lib/python3/dist-packages/obitools/ecopcr/taxonomy.py", line 420, in __init__
self._readNodeTable('%s/nodes.dmp' % taxdir)
File "/usr/lib/python3/dist-packages/obitools/ecopcr/taxonomy.py", line 496, in _readNodeTable
rankidx = dict(map(None,ranks,range(len(ranks))))
TypeError: 'NoneType' object is not callable
Exception ignored in: <function EcoPCRDBSequenceWriter.__del__ at 0x7fa33985d310>
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/obitools/ecopcr/sequence.py", line 180, in __del__
self.close()
File "/usr/lib/python3/dist-packages/obitools/ecopcr/sequence.py", line 149, in close
self._file.seek(0,0)
AttributeError: 'EcoPCRDBSequenceWriter' object has no attribute '_file'Hi everyone,
I am having this issue with obiconvert, hope someone could help me!
**obiconvert --embl -t ./TAXO --ecopcrdb-output=embl_last ./EMBL/*.dat**
Reading taxonomy dump file...
List all taxonomy rank...
Traceback (most recent call last):
File "/usr/bin/obiconvert", line 43, in <module>
writer = sequenceWriterGenerator(options)
File "/usr/lib/python3/dist-packages/obitools/format/options.py", line 364, in sequenceWriterGenerator
writer=EcoPCRDBSequenceWriter(options)
File "/usr/lib/python3/dist-packages/obitools/ecopcr/sequence.py", line 72, in __init__
self._taxonomy= loadTaxonomyDatabase(options)
File "/usr/lib/python3/dist-packages/obitools/ecopcr/options.py", line 79, in loadTaxonomyDatabase
taxonomy = TaxonomyDump(options.taxdump)
File "/usr/lib/python3/dist-packages/obitools/ecopcr/taxonomy.py", line 420, in __init__
self._readNodeTable('%s/nodes.dmp' % taxdir)
File "/usr/lib/python3/dist-packages/obitools/ecopcr/taxonomy.py", line 496, in _readNodeTable
rankidx = dict(map(None,ranks,range(len(ranks))))
TypeError: 'NoneType' object is not callable
Exception ignored in: <function EcoPCRDBSequenceWriter.__del__ at 0x7fa33985d310>
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/obitools/ecopcr/sequence.py", line 180, in __del__
self.close()
File "/usr/lib/python3/dist-packages/obitools/ecopcr/sequence.py", line 149, in close
self._file.seek(0,0)
AttributeError: 'EcoPCRDBSequenceWriter' object has no attribute '_file'https://git.metabarcoding.org/obitools/obitools/-/issues/53ngsfilter2020-01-23T09:14:13ZGabriele CananzingsfilterHello!
I'm having problems in demultiplexing my sequences with **ngsfilter**.
Essentially, the program assigns sequences just to the sample belonging to the first raw of my sample_description.txt file, ignoring all the other tag combinations below, while in the "unassigned" file I found various existing tag combinations. Furthermore, in most of the assigned sequences, **ngsfilter** is not able to assign the forward tag, the reverse tag or both tags to sequences. Here I attach two sequences taken from the output of my **ngsfilter** run.
>K00253:122:HFGGKBBXY:7:1101:16691:1226_CONS_SUB_CMP ali_length=91; seq_ab_match=91; tail_quality=36.5; reverse_match=gggtatctaatcccagtttg; seq_a_deletion=0; reverse_score=80.0; reverse_primer=gggtatctaatcccagtttg; merged_sample={'b01_1': 1}; seq_a_mismatch=0; seq_b_mismatch=0; score=363.844489406; mid_quality=61.2827225131; avg_quality=58.9336492891; seq_a_single=60; score_norm=3.99829109237; status=partial; direction=reverse; seq_b_insertion=0; experiment=1; seq_b_deletion=0; seq_a_insertion=0; seq_length_ori=211; count=1; seq_length=181; mode=alignment; head_quality=36.5; seq_b_single=60;
ntcgtgccagccaccgcggttatacgagaggcccaagttgataggtaacggcgtaaaggg
tggttagggagtactatacagtaaagccgaacgtctatcaatgttgtttgaagctctcga
agattggaagccccgccacgaaagtgactttaccatccctgaatccacgaaagccagggc
a
>K00253:122:HFGGKBBXY:7:1101:6492:1244_CONS_SUB_SUB_CMP status=full; seq_a_deletion=0; forward_primer=aaactcgtgccagccacc; reverse_primer=gggtatctaatcccagtttg; merged_sample={'b01_1': 1626}; experiment=1; seq_b_insertion=0; seq_b_deletion=0; seq_a_insertion=0; reverse_tag=acacacac; count=1626; seq_length=163; mode=alignment;
gcggttatacgaggggcccaagttgacagaagacggcgtaaagcgtggttaatgagagat
acaataaagccaaattccatcaaggctgtcatacgcatccgatgacgagaggaaccccag
cgaaagtgactttagtaagcatgaggccacgaaagctagggaa
As you can see, both sequences are assigned to the sample b01_1, which corresponds to the sample on the first raw of the sample_description.txt file. [TELE02_sample_description.txt](/uploads/8cd485bdcf1fc0dd1c5472ecf0a8c908/TELE02_sample_description.txt).
I may have mistaken something in the preceding steps, or in the text file format.
Does anybody have the same issue or any suggestion?
Thank you very much!Hello!
I'm having problems in demultiplexing my sequences with **ngsfilter**.
Essentially, the program assigns sequences just to the sample belonging to the first raw of my sample_description.txt file, ignoring all the other tag combinations below, while in the "unassigned" file I found various existing tag combinations. Furthermore, in most of the assigned sequences, **ngsfilter** is not able to assign the forward tag, the reverse tag or both tags to sequences. Here I attach two sequences taken from the output of my **ngsfilter** run.
>K00253:122:HFGGKBBXY:7:1101:16691:1226_CONS_SUB_CMP ali_length=91; seq_ab_match=91; tail_quality=36.5; reverse_match=gggtatctaatcccagtttg; seq_a_deletion=0; reverse_score=80.0; reverse_primer=gggtatctaatcccagtttg; merged_sample={'b01_1': 1}; seq_a_mismatch=0; seq_b_mismatch=0; score=363.844489406; mid_quality=61.2827225131; avg_quality=58.9336492891; seq_a_single=60; score_norm=3.99829109237; status=partial; direction=reverse; seq_b_insertion=0; experiment=1; seq_b_deletion=0; seq_a_insertion=0; seq_length_ori=211; count=1; seq_length=181; mode=alignment; head_quality=36.5; seq_b_single=60;
ntcgtgccagccaccgcggttatacgagaggcccaagttgataggtaacggcgtaaaggg
tggttagggagtactatacagtaaagccgaacgtctatcaatgttgtttgaagctctcga
agattggaagccccgccacgaaagtgactttaccatccctgaatccacgaaagccagggc
a
>K00253:122:HFGGKBBXY:7:1101:6492:1244_CONS_SUB_SUB_CMP status=full; seq_a_deletion=0; forward_primer=aaactcgtgccagccacc; reverse_primer=gggtatctaatcccagtttg; merged_sample={'b01_1': 1626}; experiment=1; seq_b_insertion=0; seq_b_deletion=0; seq_a_insertion=0; reverse_tag=acacacac; count=1626; seq_length=163; mode=alignment;
gcggttatacgaggggcccaagttgacagaagacggcgtaaagcgtggttaatgagagat
acaataaagccaaattccatcaaggctgtcatacgcatccgatgacgagaggaaccccag
cgaaagtgactttagtaagcatgaggccacgaaagctagggaa
As you can see, both sequences are assigned to the sample b01_1, which corresponds to the sample on the first raw of the sample_description.txt file. [TELE02_sample_description.txt](/uploads/8cd485bdcf1fc0dd1c5472ecf0a8c908/TELE02_sample_description.txt).
I may have mistaken something in the preceding steps, or in the text file format.
Does anybody have the same issue or any suggestion?
Thank you very much!https://git.metabarcoding.org/obitools/obitools/-/issues/30Missing shebang in extractreads.py and extractreads2.py2020-01-18T18:47:48ZPhilippe BordronMissing shebang in extractreads.py and extractreads2.pyThe files `extractreads.py` and `extractreads2.py` miss the shebang.
Consequences are:
* `extractreads.py` and `extractreads2.py` do not work when installing with 'python setup.py install'
* help switch (-h or --help) doesn't work for `extractreads.py` and `extractreads2.py`
Adding the following shebang at the start of `extractreads.py` and `extractreads2.py` fixes the issue:
```
#!/usr/local/bin/python
```
Edit: related patches
[extractreads.patch](/uploads/78e8005a3940443aadf10689d4a78fdf/extractreads.patch)
[extractreads2.patch](/uploads/13d8d816ac4b332708209b1b05f64497/extractreads2.patch)The files `extractreads.py` and `extractreads2.py` miss the shebang.
Consequences are:
* `extractreads.py` and `extractreads2.py` do not work when installing with 'python setup.py install'
* help switch (-h or --help) doesn't work for `extractreads.py` and `extractreads2.py`
Adding the following shebang at the start of `extractreads.py` and `extractreads2.py` fixes the issue:
```
#!/usr/local/bin/python
```
Edit: related patches
[extractreads.patch](/uploads/78e8005a3940443aadf10689d4a78fdf/extractreads.patch)
[extractreads2.patch](/uploads/13d8d816ac4b332708209b1b05f64497/extractreads2.patch)https://git.metabarcoding.org/obitools/obitools/-/issues/52error message in obicomplement2019-12-12T10:16:04ZLucie Zingererror message in obicomplementHi, on the obitools version obitools-v1.2.11, I get the following error message when running obicomplement:
$ obicomplement locDB_Funguy2019.fasta > locDB_Funguy2019_RC.fasta
locDB_Funguy2019.fasta 57.8 % |############################| ] remain : 00:00:00
Traceback (most recent call last):
File "/usr/local/bioinfo/src/OBITools/obitools-v1.2.11/venv_obitools-v1.2.11/bin/obicomplement", line 58, in <module>
writer(seq.complement())
AttributeError: 'obitools._obitools.AASequence' object has no attribute 'complement'
The output however does not seems corrupted or truncated. Any idea of what would be the problem here? thanks. LucieHi, on the obitools version obitools-v1.2.11, I get the following error message when running obicomplement:
$ obicomplement locDB_Funguy2019.fasta > locDB_Funguy2019_RC.fasta
locDB_Funguy2019.fasta 57.8 % |############################| ] remain : 00:00:00
Traceback (most recent call last):
File "/usr/local/bioinfo/src/OBITools/obitools-v1.2.11/venv_obitools-v1.2.11/bin/obicomplement", line 58, in <module>
writer(seq.complement())
AttributeError: 'obitools._obitools.AASequence' object has no attribute 'complement'
The output however does not seems corrupted or truncated. Any idea of what would be the problem here? thanks. Luciehttps://git.metabarcoding.org/obitools/obitools/-/issues/51ecoPCR freezes without finishing.2019-11-19T10:17:40ZBrian GillecoPCR freezes without finishing.Hi there.
I'm trying to run ecoPCR on a file of 4,987 unique sequences.
My reference database, generated from ENA, contains 20,926 sequences between 9-271 bp with a median around 50 bp.
The program freezes shortly after starting. The progress bar is stuck at 0.2% and the timer indicates that there is no time left until the program is done.
Looking at the activity monitor on my Mac though it seems to continue to run and to be using increasing amounts of memory (up to ~24 gigs).
Is it just too big of a job?
Everything seems to work fine when I query these same 4,987 sequences against a smaller reference database of 307 sequences.
Files attached.
Thanks.
Brian[all_uniq.c10.l5.clean.fasta](/uploads/ca7647fcecc00b9f59c3db38cd94bc0a/all_uniq.c10.l5.clean.fasta)[db_v141.fasta.zip](/uploads/1a5efcaf6dd810386e942113a04d26bc/db_v141.fasta.zip)
Running: ecoTag -t Taxonomy.db/taxdump/ -R db_v141.fasta all_uniq.c10.l5.clean.fasta > all_uniq.c10.l5.clean.tag.fastaHi there.
I'm trying to run ecoPCR on a file of 4,987 unique sequences.
My reference database, generated from ENA, contains 20,926 sequences between 9-271 bp with a median around 50 bp.
The program freezes shortly after starting. The progress bar is stuck at 0.2% and the timer indicates that there is no time left until the program is done.
Looking at the activity monitor on my Mac though it seems to continue to run and to be using increasing amounts of memory (up to ~24 gigs).
Is it just too big of a job?
Everything seems to work fine when I query these same 4,987 sequences against a smaller reference database of 307 sequences.
Files attached.
Thanks.
Brian[all_uniq.c10.l5.clean.fasta](/uploads/ca7647fcecc00b9f59c3db38cd94bc0a/all_uniq.c10.l5.clean.fasta)[db_v141.fasta.zip](/uploads/1a5efcaf6dd810386e942113a04d26bc/db_v141.fasta.zip)
Running: ecoTag -t Taxonomy.db/taxdump/ -R db_v141.fasta all_uniq.c10.l5.clean.fasta > all_uniq.c10.l5.clean.tag.fastahttps://git.metabarcoding.org/obitools/obitools/-/issues/49Could not import sequence id: AE005172; (error raised: 'NoneType' object has ...2019-10-06T11:56:44ZMarc-Olivier BeausoleilCould not import sequence id: AE005172; (error raised: 'NoneType' object has no attribute 'group' )I'm trying to make a reference database for trnL markers and I have an error:
This is the command I run:
```
obiconvert --embl -t $TAXO --ecopcrdb-output=embl_6617 $EMBL/*.dat.gz
```
$TAXO is a path that leads to the data found here: ftp://ftp.ncbi.nih.gov/pub/taxonomy/taxdump.tar.gz
$EMBL/*.dat.gz is leading to the .dat.gz file (I'm trying only one to see what it would do: this one rel_con_pln_01_r141.dat.gz)
Then it starts:
```
Reading taxonomy dump file...
List all taxonomy rank...
Indexing taxonomy...
Indexing parent and rank...
Adding scientific name...
Adding taxid alias...
Adding deleted taxid...
Writing the taxonomy file... Ok
```
And throw this error:
```
Could not import sequence id: AE005172; (error raised: 'NoneType' object has no attribute 'group' )
Traceback (most recent call last):
File "/usr/local/bin/obiconvert", line 45, in <module>
for entry in entries:
File "src/obitools/options/_options.pyx", line 104, in allEntryIterator
File "/usr/local/lib/python2.7/site-packages/obitools/format/options.py", line 212, in iterator
s.extractTaxon()
AttributeError: 'NoneType' object has no attribute 'extractTaxon'
```
I looked at the AE005172 sequence (https://www.ncbi.nlm.nih.gov/nuccore/AE005172.1) and it seems to be "Arabidopsis thaliana chromosome 1 top arm, complete sequence".
But I have no idea on what to do now. How should I deal with this error?I'm trying to make a reference database for trnL markers and I have an error:
This is the command I run:
```
obiconvert --embl -t $TAXO --ecopcrdb-output=embl_6617 $EMBL/*.dat.gz
```
$TAXO is a path that leads to the data found here: ftp://ftp.ncbi.nih.gov/pub/taxonomy/taxdump.tar.gz
$EMBL/*.dat.gz is leading to the .dat.gz file (I'm trying only one to see what it would do: this one rel_con_pln_01_r141.dat.gz)
Then it starts:
```
Reading taxonomy dump file...
List all taxonomy rank...
Indexing taxonomy...
Indexing parent and rank...
Adding scientific name...
Adding taxid alias...
Adding deleted taxid...
Writing the taxonomy file... Ok
```
And throw this error:
```
Could not import sequence id: AE005172; (error raised: 'NoneType' object has no attribute 'group' )
Traceback (most recent call last):
File "/usr/local/bin/obiconvert", line 45, in <module>
for entry in entries:
File "src/obitools/options/_options.pyx", line 104, in allEntryIterator
File "/usr/local/lib/python2.7/site-packages/obitools/format/options.py", line 212, in iterator
s.extractTaxon()
AttributeError: 'NoneType' object has no attribute 'extractTaxon'
```
I looked at the AE005172 sequence (https://www.ncbi.nlm.nih.gov/nuccore/AE005172.1) and it seems to be "Arabidopsis thaliana chromosome 1 top arm, complete sequence".
But I have no idea on what to do now. How should I deal with this error?https://git.metabarcoding.org/obitools/obitools/-/issues/38Error in building the reference database2019-10-02T02:38:12ZHarsh ShuklaError in building the reference databaseHi,
Thank you for the wonderful pipeline. I recently downloaded the EMBL release 138 dataset and was trying to build the reference database. But while running the command :
obiconvert --embl -t ./TAXO --ecopcrDB-output=embl_last ./EMBL/*.dat
I encountered an error. The last few lines of log indicating the error are as follows.
Error on sequence : EU394480
Traceback (most recent call last):
File "/home/uramakri/mousumig/Softwares/OBITools/OBITools-1.2.12/export/bin/obiconvert", line 52, in <module>
writer(entry)
File "/home/uramakri/mousumig/Softwares/OBITools/OBITools-1.2.12/lib/python2.7/site-packages/obitools/format/options.py", line 371, in sequen
writer.put(sequence)
File "/home/uramakri/mousumig/Softwares/OBITools/OBITools-1.2.12/lib/python2.7/site-packages/obitools/ecopcr/sequence.py", line 173, in put
self._file.write(self._ecoSeqPacker(sequence))
File "/home/uramakri/mousumig/Softwares/OBITools/OBITools-1.2.12/lib/python2.7/site-packages/obitools/ecopcr/sequence.py", line 141, in _ecoS
raise e
struct.error: cannot convert argument to integer
I am also attaching the log for the last obiconvert run.
[obiconvert.err](/uploads/aaa8fcfd20dc785ea84e8aa9e60c285c/obiconvert.err)
I would be quite grateful if someone could help me troubleshoot the error.
Thanking You
With Regards,
HarshHi,
Thank you for the wonderful pipeline. I recently downloaded the EMBL release 138 dataset and was trying to build the reference database. But while running the command :
obiconvert --embl -t ./TAXO --ecopcrDB-output=embl_last ./EMBL/*.dat
I encountered an error. The last few lines of log indicating the error are as follows.
Error on sequence : EU394480
Traceback (most recent call last):
File "/home/uramakri/mousumig/Softwares/OBITools/OBITools-1.2.12/export/bin/obiconvert", line 52, in <module>
writer(entry)
File "/home/uramakri/mousumig/Softwares/OBITools/OBITools-1.2.12/lib/python2.7/site-packages/obitools/format/options.py", line 371, in sequen
writer.put(sequence)
File "/home/uramakri/mousumig/Softwares/OBITools/OBITools-1.2.12/lib/python2.7/site-packages/obitools/ecopcr/sequence.py", line 173, in put
self._file.write(self._ecoSeqPacker(sequence))
File "/home/uramakri/mousumig/Softwares/OBITools/OBITools-1.2.12/lib/python2.7/site-packages/obitools/ecopcr/sequence.py", line 141, in _ecoS
raise e
struct.error: cannot convert argument to integer
I am also attaching the log for the last obiconvert run.
[obiconvert.err](/uploads/aaa8fcfd20dc785ea84e8aa9e60c285c/obiconvert.err)
I would be quite grateful if someone could help me troubleshoot the error.
Thanking You
With Regards,
Harshhttps://git.metabarcoding.org/obitools/obitools/-/issues/37ecotag: Assertion Error2019-05-09T14:58:25ZCharles Baillieecotag: Assertion ErrorHi Folks,
I keep getting the following error when running ecotag:
File "/home/els/SOFTWARE/OBITools-1.2.12/bin/ecotag", line 345, in <module>
assert seqid not in taxonlink
AssertionError
Any ideas?
Thanks,
CharlesHi Folks,
I keep getting the following error when running ecotag:
File "/home/els/SOFTWARE/OBITools-1.2.12/bin/ecotag", line 345, in <module>
assert seqid not in taxonlink
AssertionError
Any ideas?
Thanks,
Charleshttps://git.metabarcoding.org/obitools/obitools/-/issues/41Error when running Ecotag2019-05-09T14:57:53ZTania Valdivia-CarrilloError when running EcotagI am getting an error when I try to run ecotag, the error goes like:
```
MacBook: taniavc$ ecotag -t ~/data/TAXA/nodes -R ~/data/DABA/Teleostei_mito.fasta S1_seeds_nonsingleton.fasta > S1.ecotag.fasta
Reading taxonomy dump file...
List all taxonomy rank...
Indexing taxonomy...
Indexing parent and rank...
Adding scientific name...
Adding taxid alias...
Adding deleted taxid...
Reading reference DB ... : 304600
Traceback (most recent call last):
File "/Users/taniavc/miniconda3/bin/ecotag", line 346, in <module>
taxonlink[seqid]=int(seq['taxid'])
File "src/obitools/_obitools.pyx", line 263, in obitools._obitools.BioSequence.__getitem__
File "src/obitools/_obitools.pyx", line 217, in obitools._obitools.BioSequence.getKey
KeyError: 'taxid'
```
Info: Inside ~/data/TAXA/nodes I have: citations.dmp, division.dmp, merged.dmp, nodes.dmp, delnodes.dmp, gencode.dmp, names.dmp. I constructed my DB from Genbank, and its a Fasta file. My S1_seeds_nonsingleton.fasta is the result of the previous Obitools steps.
Any ideas for what I am doing wrong? Thanks!I am getting an error when I try to run ecotag, the error goes like:
```
MacBook: taniavc$ ecotag -t ~/data/TAXA/nodes -R ~/data/DABA/Teleostei_mito.fasta S1_seeds_nonsingleton.fasta > S1.ecotag.fasta
Reading taxonomy dump file...
List all taxonomy rank...
Indexing taxonomy...
Indexing parent and rank...
Adding scientific name...
Adding taxid alias...
Adding deleted taxid...
Reading reference DB ... : 304600
Traceback (most recent call last):
File "/Users/taniavc/miniconda3/bin/ecotag", line 346, in <module>
taxonlink[seqid]=int(seq['taxid'])
File "src/obitools/_obitools.pyx", line 263, in obitools._obitools.BioSequence.__getitem__
File "src/obitools/_obitools.pyx", line 217, in obitools._obitools.BioSequence.getKey
KeyError: 'taxid'
```
Info: Inside ~/data/TAXA/nodes I have: citations.dmp, division.dmp, merged.dmp, nodes.dmp, delnodes.dmp, gencode.dmp, names.dmp. I constructed my DB from Genbank, and its a Fasta file. My S1_seeds_nonsingleton.fasta is the result of the previous Obitools steps.
Any ideas for what I am doing wrong? Thanks!https://git.metabarcoding.org/obitools/obitools/-/issues/40Issue when running obiconvert -d ncbi --ecopcrdb-output2019-05-09T14:57:26ZAlejandro Maeda ObregonIssue when running obiconvert -d ncbi --ecopcrdb-outputHello, I am currently using Obitools on a Mac to make my primers, but I have been getting to this issue in which something happens with the code and the taxaid
obiconvert -d ncbi --ecopcrdb-output=mito.insecta mito.insecta.fasta
Reading binary taxonomy database...
[INFO : Taxon alias file found]
[INFO : Local taxon file not found]
Taxonomical tree read
[Errno 2] No such file or directory: 'ncbi.pdx'
[INFO : Preferred taxon name file not found]
ok
Writing the taxonomy file... Ok
mito.insecta.fasta 0.5 % |/ ] remain : 00:00:00Traceback (most recent call last):
File "/miniconda2/bin/obiconvert", line 52, in <module>
writer(entry)
File "/miniconda2/lib/python2.7/site-packages/obitools/format/options.py", line 371, in sequenceWriter
writer.put(sequence)
File "/miniconda2/lib/python2.7/site-packages/obitools/ecopcr/sequence.py", line 173, in put
self._file.write(self._ecoSeqPacker(sequence))
File "/miniconda2/lib/python2.7/site-packages/obitools/ecopcr/sequence.py", line 127, in _ecoSeqPacker
raise Exception("Taxonomy error for %s: %s"%(seq.id, "taxonomy is missing" if self._taxonomy is None else "sequence has no taxid" if 'taxid' not in seq else "wrong taxid"))
Exception: Taxonomy error for MF925724.1: sequence has no taxid
Has someone encountered this issue before? I have no idea how to get around this. I also tried to uninstall all packages but that does not solve anything. Any help would be appreciated.Hello, I am currently using Obitools on a Mac to make my primers, but I have been getting to this issue in which something happens with the code and the taxaid
obiconvert -d ncbi --ecopcrdb-output=mito.insecta mito.insecta.fasta
Reading binary taxonomy database...
[INFO : Taxon alias file found]
[INFO : Local taxon file not found]
Taxonomical tree read
[Errno 2] No such file or directory: 'ncbi.pdx'
[INFO : Preferred taxon name file not found]
ok
Writing the taxonomy file... Ok
mito.insecta.fasta 0.5 % |/ ] remain : 00:00:00Traceback (most recent call last):
File "/miniconda2/bin/obiconvert", line 52, in <module>
writer(entry)
File "/miniconda2/lib/python2.7/site-packages/obitools/format/options.py", line 371, in sequenceWriter
writer.put(sequence)
File "/miniconda2/lib/python2.7/site-packages/obitools/ecopcr/sequence.py", line 173, in put
self._file.write(self._ecoSeqPacker(sequence))
File "/miniconda2/lib/python2.7/site-packages/obitools/ecopcr/sequence.py", line 127, in _ecoSeqPacker
raise Exception("Taxonomy error for %s: %s"%(seq.id, "taxonomy is missing" if self._taxonomy is None else "sequence has no taxid" if 'taxid' not in seq else "wrong taxid"))
Exception: Taxonomy error for MF925724.1: sequence has no taxid
Has someone encountered this issue before? I have no idea how to get around this. I also tried to uninstall all packages but that does not solve anything. Any help would be appreciated.https://git.metabarcoding.org/obitools/obitools/-/issues/42ngsfilter: AssertionError: tag pair2019-05-09T14:56:30ZOmer Golanngsfilter: AssertionError: tag pairHello,
I am getting this error when trying to run the ngsfilter command-
"Traceback (most recent call last):
File "/mnt/c/Users/Omer/Desktop/OBItools/OBITools-1.2.13/export/bin/ngsfilter", line 426, in <module>
primers=readTagfile(options.taglist)
File "/mnt/c/Users/Omer/Desktop/OBItools/OBITools-1.2.13/export/bin/ngsfilter", line 227, in readTagfile
"tag pair %s is already used with primer pairs : (%s,%s)" % (str(tags),forward,reverse)
AssertionError: tag pair ('atgc', 'atgc') is already used with primer pairs : (D: ttcatctcagactgggattcagaaaggc,R: gcttggaaataaccctcctgcatccc)"
This is the command I wrote-
"ngsfilter -t rawdata_GZ_2-22.ngsfilter.txt -u rawdata_GZ_2-22.unidentified.fastq rawdata_GZ_2-22.Alignement.fastq > rawdata_GZ_2-22.filtered.fastq"
Am I doing anything wrong?
Thanks..Hello,
I am getting this error when trying to run the ngsfilter command-
"Traceback (most recent call last):
File "/mnt/c/Users/Omer/Desktop/OBItools/OBITools-1.2.13/export/bin/ngsfilter", line 426, in <module>
primers=readTagfile(options.taglist)
File "/mnt/c/Users/Omer/Desktop/OBItools/OBITools-1.2.13/export/bin/ngsfilter", line 227, in readTagfile
"tag pair %s is already used with primer pairs : (%s,%s)" % (str(tags),forward,reverse)
AssertionError: tag pair ('atgc', 'atgc') is already used with primer pairs : (D: ttcatctcagactgggattcagaaaggc,R: gcttggaaataaccctcctgcatccc)"
This is the command I wrote-
"ngsfilter -t rawdata_GZ_2-22.ngsfilter.txt -u rawdata_GZ_2-22.unidentified.fastq rawdata_GZ_2-22.Alignement.fastq > rawdata_GZ_2-22.filtered.fastq"
Am I doing anything wrong?
Thanks..https://git.metabarcoding.org/obitools/obitools/-/issues/39ecoTag hanging2019-02-11T15:37:09ZPatrick FreemanecoTag hangingHi there,
I am currently trying to build a trnL reference library from the EMBL Release 130 for a plant metabarcoding project. I have been trying to follow the tutorial outlined on the Obitools webpage but I've been running into several issues.
One issue I'm having is with the taxonomy assignment using the ecoTag function. Every time I try to run the ecoTag program, it only goes to about 74% completion and then the time reads out 00:00:00 and the cache size is printed. I've tried increasing the cache size by several orders of magnitude but to no avail. Am I missing something in what could be causing this part of the program to hang?
Code below:
ecotag -d ./GlobalRef/EMBL_Taxo/R130/embl_r130 -R ./GlobalRef/gh_database_r130.fasta --sort=count -m 0.98 -r physeq_merged_count_sort.fasta > physeq_merged_count_sort_ecotagGlobal_r130.fastaHi there,
I am currently trying to build a trnL reference library from the EMBL Release 130 for a plant metabarcoding project. I have been trying to follow the tutorial outlined on the Obitools webpage but I've been running into several issues.
One issue I'm having is with the taxonomy assignment using the ecoTag function. Every time I try to run the ecoTag program, it only goes to about 74% completion and then the time reads out 00:00:00 and the cache size is printed. I've tried increasing the cache size by several orders of magnitude but to no avail. Am I missing something in what could be causing this part of the program to hang?
Code below:
ecotag -d ./GlobalRef/EMBL_Taxo/R130/embl_r130 -R ./GlobalRef/gh_database_r130.fasta --sort=count -m 0.98 -r physeq_merged_count_sort.fasta > physeq_merged_count_sort_ecotagGlobal_r130.fastahttps://git.metabarcoding.org/obitools/obitools/-/issues/21Obiconvert error (in the _ecoSeqPacker function)2019-01-18T09:32:48ZCeline MercierObiconvert error (in the _ecoSeqPacker function)Someone is getting an error using obiconvert:
"Trying to make a reference database for trnL for plants. I get an error when using obiconvert (see below). Is it serious or can I just ignore?
(my embl_last_001.sdx is 49 GB, so something has happened…)"
```
#Format the data
mkdir embl_database
cd embl_database
nice -n 19 obiconvert --embl -t ../TAXO --ecopcrdb-output=embl_last ../EMBL/*.dat.gz
#But I get an error message
File "/usr/local/bin/obiconvert", line 52, in <module>
writer(entry)
File "/usr/local/lib/python2.7/dist-packages/obitools/format/options.py", line 349, in sequenceWriter
writer.put(sequence)
File "/usr/local/lib/python2.7/dist-packages/obitools/ecopcr/sequence.py", line 148, in put
self._file.write(self._ecoSeqPacker(sequence))
File "/usr/local/lib/python2.7/dist-packages/obitools/ecopcr/sequence.py", line 137, in _ecoSeqPacker
raise e
struct.error: cannot convert argument to integer
```Someone is getting an error using obiconvert:
"Trying to make a reference database for trnL for plants. I get an error when using obiconvert (see below). Is it serious or can I just ignore?
(my embl_last_001.sdx is 49 GB, so something has happened…)"
```
#Format the data
mkdir embl_database
cd embl_database
nice -n 19 obiconvert --embl -t ../TAXO --ecopcrdb-output=embl_last ../EMBL/*.dat.gz
#But I get an error message
File "/usr/local/bin/obiconvert", line 52, in <module>
writer(entry)
File "/usr/local/lib/python2.7/dist-packages/obitools/format/options.py", line 349, in sequenceWriter
writer.put(sequence)
File "/usr/local/lib/python2.7/dist-packages/obitools/ecopcr/sequence.py", line 148, in put
self._file.write(self._ecoSeqPacker(sequence))
File "/usr/local/lib/python2.7/dist-packages/obitools/ecopcr/sequence.py", line 137, in _ecoSeqPacker
raise e
struct.error: cannot convert argument to integer
```https://git.metabarcoding.org/obitools/obitools/-/issues/34Why isn't 'obiclean_headcount > 0' the same as 'obiclean_head == True'?2018-11-19T15:04:57ZDani Díaz de QuijanoWhy isn't 'obiclean_headcount > 0' the same as 'obiclean_head == True'?I run obiclean on my lake water protist community data set without the option -H.
(I do it manually because after obiclean I would like to divide the dataset into abundant uniqs and rare ones. I will take only heads in the case of abundant community but I plan to keep singletons as well to explore the rare protist community, i.e. just to remove internals, using obigrep -p 'obiclean_internalcount == 0'.)
I understand the option -H Selects only sequences with the head status in a least one sample, the same way as using obigrep -p 'obiclean_head == True'. I got the same results doing it in both ways. My concern is maybe I didn't understand obiclean_head annotation or maybe there is a mistake. I understand and corroborated obiclean_head=True when at least one of the samples in obiclean_status is labelled 'h', and accordingly obiclean_headcount > 0. BUT I checked My fasta file and there are some sequences where I have e.g.:
obiclean_status={'Nov1B': 'i', 'Sep2B': 's'}
obiclean_internalcount=1
obiclean_singletoncount=1
obiclean_headcount=0
BUT, surprisingly:
obiclean_head=True
It is not an isolated case:
In my abundant sequences I get:
1883 unique sequences and 1226761 reads if I use obigrep -p 'obiclean_head == True', and I get
1264 unique sequences and 1210278 reads if I use obigrep -p 'obiclean_headcount > 0'
In my rare sequences I get:
6937 unique sequences and 24307 reads if I use obigrep -p 'obiclean_head == True', and I get
700 unique sequences and 3749 reads if I use obigrep -p 'obiclean_headcount > 0'
Why isn't 'obiclean_headcount > 0' the same as 'obiclean_head == True'? Is it safe to use 'obiclean_head == True' or the -H obiclean option to pick only heads up?
Thank you for your attention!I run obiclean on my lake water protist community data set without the option -H.
(I do it manually because after obiclean I would like to divide the dataset into abundant uniqs and rare ones. I will take only heads in the case of abundant community but I plan to keep singletons as well to explore the rare protist community, i.e. just to remove internals, using obigrep -p 'obiclean_internalcount == 0'.)
I understand the option -H Selects only sequences with the head status in a least one sample, the same way as using obigrep -p 'obiclean_head == True'. I got the same results doing it in both ways. My concern is maybe I didn't understand obiclean_head annotation or maybe there is a mistake. I understand and corroborated obiclean_head=True when at least one of the samples in obiclean_status is labelled 'h', and accordingly obiclean_headcount > 0. BUT I checked My fasta file and there are some sequences where I have e.g.:
obiclean_status={'Nov1B': 'i', 'Sep2B': 's'}
obiclean_internalcount=1
obiclean_singletoncount=1
obiclean_headcount=0
BUT, surprisingly:
obiclean_head=True
It is not an isolated case:
In my abundant sequences I get:
1883 unique sequences and 1226761 reads if I use obigrep -p 'obiclean_head == True', and I get
1264 unique sequences and 1210278 reads if I use obigrep -p 'obiclean_headcount > 0'
In my rare sequences I get:
6937 unique sequences and 24307 reads if I use obigrep -p 'obiclean_head == True', and I get
700 unique sequences and 3749 reads if I use obigrep -p 'obiclean_headcount > 0'
Why isn't 'obiclean_headcount > 0' the same as 'obiclean_head == True'? Is it safe to use 'obiclean_head == True' or the -H obiclean option to pick only heads up?
Thank you for your attention!https://git.metabarcoding.org/obitools/obitools/-/issues/1ecoPCR help2018-01-17T17:39:19ZAurélie BoninecoPCR helpSomething that should be easy to fix:
the ecoPCR -D option doesn't appear on screen when typing "ecoPCR -h", but it can be found in the online help (http://metabarcoding.org//obitools/doc/scripts/ecoPCR.html?highlight=ecopcr)Something that should be easy to fix:
the ecoPCR -D option doesn't appear on screen when typing "ecoPCR -h", but it can be found in the online help (http://metabarcoding.org//obitools/doc/scripts/ecoPCR.html?highlight=ecopcr)Eric CoissacEric Coissachttps://git.metabarcoding.org/obitools/obitools/-/issues/3--rank should be --seq_rank in obiannotate2018-01-17T17:39:19ZGhost User--rank should be --seq_rank in obiannotateDans le -h, c'est --seq_rank, mais dans le tutorial c'est --rank.
--seq_rank et l'option qui marche.
PierreDans le -h, c'est --seq_rank, mais dans le tutorial c'est --rank.
--seq_rank et l'option qui marche.
Pierrehttps://git.metabarcoding.org/obitools/obitools/-/issues/5Branch names2017-09-29T05:28:39ZEric CoissacBranch namesThe branches in the obitools repository is inherited for the old subversion repository and do not reflect any more the development of obitools. The main development are done now on branch `obitools-1.01` and the `master` branch is a dead branch without any development since two year.
I think we have to rename the branch `obitools-1.01` `master` and the formal `master` branch `dead something` :smiley: . But after this renaming we will have to fully re-checkout the full project because of the branch history discontinuity.The branches in the obitools repository is inherited for the old subversion repository and do not reflect any more the development of obitools. The main development are done now on branch `obitools-1.01` and the `master` branch is a dead branch without any development since two year.
I think we have to rename the branch `obitools-1.01` `master` and the formal `master` branch `dead something` :smiley: . But after this renaming we will have to fully re-checkout the full project because of the branch history discontinuity.https://git.metabarcoding.org/obitools/obitools/-/issues/6Installation problem2017-09-29T05:28:39ZCeline MercierInstallation problemWhen I use the ``get-obitools.py`` script to install the OBITools, when I try running ``obitools``, I get:
```
./obitools
./obitools: line 9: /Users/celinemercier/obitests/OBITools-1.1.16/bin/pidname: No such file or directory
You can now use OBITools
./obitools: line 34: -c: command not found
OBITools are no more activated, Bye...
```When I use the ``get-obitools.py`` script to install the OBITools, when I try running ``obitools``, I get:
```
./obitools
./obitools: line 9: /Users/celinemercier/obitests/OBITools-1.1.16/bin/pidname: No such file or directory
You can now use OBITools
./obitools: line 34: -c: command not found
OBITools are no more activated, Bye...
```https://git.metabarcoding.org/obitools/obitools/-/issues/7Another installation problem, with setup.py2017-09-29T05:28:39ZCeline MercierAnother installation problem, with setup.pyI'm not sure why but I had to remove one indentation level in ``distutils.ext/obidistutils/command/build_sphinx.py``, lines 21 to 37, otherwise I got:
```
python setup.py build
Traceback (most recent call last):
File "setup.py", line 39, in <module>
from obidistutils.core import setup
File "distutils.ext/obidistutils/core.py", line 27, in <module>
from obidistutils.command.build_sphinx import build_sphinx
ImportError: cannot import name build_sphinx
```
Should I commit?I'm not sure why but I had to remove one indentation level in ``distutils.ext/obidistutils/command/build_sphinx.py``, lines 21 to 37, otherwise I got:
```
python setup.py build
Traceback (most recent call last):
File "setup.py", line 39, in <module>
from obidistutils.core import setup
File "distutils.ext/obidistutils/core.py", line 27, in <module>
from obidistutils.command.build_sphinx import build_sphinx
ImportError: cannot import name build_sphinx
```
Should I commit?