Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
O
OBITools3
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 33
    • Issues 33
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Merge Requests 0
    • Merge Requests 0
  • Operations
    • Operations
    • Incidents
  • Analytics
    • Analytics
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
  • OBITools
  • OBITools3
  • Issues
  • #135

Closed
Open
Opened 1 year ago by Thomas D Hughes@tdhughes18
  • New issue

  • Report abuse

  • New issue

  • Report abuse

EMBL importing issue

Open

EMBL importing issue

Hi,

I have an issue which seems to be similar or the same as issue #129.

I am trying to import the latest EMBL release using the methods outlined in the wolf tutorial but I get this error below. on this file Parsing this file STD_PLN_4.dat.gz. I have tired removing the file to see if the issue only this file is the issue but I also get the fault with STD_VRL_11.dat.gz.

I have tried updating obitools using pip and currently have version 3.0.1b22 which I hope is the latest version but it doesn't seem to resolve the issue for me at least.

The sequence that cannot be imported is very long so I just attached the start and the end which I think are the only bits which may have useful information.

DEBUG /private/var/folders/hx/z6wbn36d04l48hw501jt40lm0000gn/T/pip-install-gxjlmvbt/obitools3_e27c130ed86d4de0aeeea00f0794a369/src/obiavl.c:1669:obi_create_avl, obi_errno = 20, errno = 24 : Error creating an AVL tree file DEBUG /private/var/folders/hx/z6wbn36d04l48hw501jt40lm0000gn/T/pip-install-gxjlmvbt/obitools3_e27c130ed86d4de0aeeea00f0794a369/src/obiavl.c:1013:add_new_avl_in_group, obi_errno = 20, errno = 24 : Error creating a new AVL tree in a group

Could not import sequence: {b'ID': b'LR812263', b'NUC_SEQ': b'CAAATCTAGCAAGAATAGGCCAAAACTACAAGTTTTGAGGAGTTCCCCGTAACTGGACCCCGAGGTTCCCGAAATGTTTGGATCACAGCGGGACACTAAATCAGTGACTAATAACATACAAAATTGTCTGCAGTAGTCCTAAACTGCGAGTTTTGACGAGTTCCACGTAACCGGACCCCGAGGTTCCTGAAACATCCGGATCGTAGCGGGACCCAAAATCAATGAGTAATTGCATACAAAACTGGCAAGAACAGGCCAAAACTGCGAGTTTTAACGAGTTTTCGGTAACCGGACCCCGGGGGTCCCGAAACATTCGTATCGCAGGGAAACCAAAAACAGTGACTAATGGCCTAGAAAACTAGCCAGAACAGGCCAACACTGTGAGTTTTCACGAATTCCCCGTAACCCCACCTCGGGGTTCCCGAAACGTTCGGATCGCAGCGGGACCCAAAATGAGTGAGTAATAGCATATAAAACTAGCCAGAATAGGCCAAAACTGCGAGTTTGAAGAGTTCCCCGTAACCACACCCCGGGGTTCCCGAAATGTTCGGATCGTAGTAGGACCCCAAATCAGTCAGTAATAGCATACAAATCTAGCAAGAATAGGCCAAAACTACAAGTTTTGAGGAGTTCCCCGTAACTGGACCCCGAG

.........

GACACGCTCGTCGGACACCGGCAGGGCAGCTAGCTAGTCTGTGCGCGTCGCGACTCCCTTCACCAGCCGTCTTCTTCGTCAACGCTCGCAAGTTGTTCGACGGTTTGCCAAGGTACAAAATGGACTCCGTCGACGAGTTCTTTTTTCACAATTTCCTTTGCGACTCCGACGATTCGTCATCCGATGACGAGGAGGAGGTATTGGCTGCCGTGTTGGTCCATCACCTGCTCAATAGCTAGCGGCCGTTGTTCCGTGGCTCCATTCCGAGCCACCTTCCGGTGTTGA', b'DEFINITION': b'Hordeum vulgare subsp. vulgare genome assembly, chromosome: 3H', b'TAXID': 112509, b'organism': b'Hordeum vulgare subsp. vulgare'} Error raised: Problem setting a value in a column /!\ Check if '--input-na-string' option needs to be set zsh: segmentation fault obi import --embl-input EMBL Guate/embl_refs

I hope I have included the relevant info, and any help would be greatly appreciated!

Best,

Tom

Please solve the reCAPTCHA

We want to be sure it is you, please confirm you are not a robot.

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information

Linked issues
...

    Related merge requests

    • Thomas D Hughes
      Thomas D Hughes @tdhughes18 · 1 year ago

      I also have an issue with importing things into obitools more generally no matter the file type it will not import even the tutorial files whether they be the fastq or the nfs files. I don't know if the issues are related or if I am doing something fundamentally wrong.

      2023-08-24 16:32:14,445 [import : INFO ] Opened file: wolf_tutorial/wolf_F.fastq 2023-08-24 16:32:14,470 [import : INFO ] Importing 45276 entries 72.4 % |####################################| ] remain : 00:00:0StopIteration

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last): File "/Users/thomashughes/obi3-env/bin/obi", line 62, in config[root_config_name]['module'].run(config) File "python/obitools3/commands/import.pyx", line 333, in obitools3.commands.import.run File "python/obitools3/parsers/fastq.pyx", line 106, in fastqWithQualityIterator RuntimeError: generator raised StopIteration

      I hope that is helpful and not making things more complicated!

      Tom

    • Timothy Divoll
      Timothy Divoll @timothy_divoll · 1 year ago

      I also get the same error @tdhughes18. Trying to import demultiplexed files in a for loop, each one hits the RuntimeError: generator raised StopIteration. Interestingly, if I use --only flag for a hard cutoff (less than the max number of entries), the seqs appear to import.

      Tim

      Edited by Timothy Divoll 1 year ago
    • Celine Mercier
      Celine Mercier @mercier · 1 year ago
      Owner

      Hi, the StopIteration issue was caused by a change in the behaviour of StopIteration exceptions in Python>=3.7, it should be fixed in the last version: https://git.metabarcoding.org/obitools/obitools3/-/wikis/Updating-and-release-notes#release-notes

      I'm also looking into the 'too many files open' error that some people run into, but it's a tricky one.

      Cheers Celine

    • Thomas D Hughes
      Thomas D Hughes @tdhughes18 · 1 year ago

      Hi, Thank you for solving the Stopiteration issue that's solved with the update for me. As for the other issue thanks for having a look at it hopefully it is possible to solve at some point!

      Tom

    • You're only seeing other activity in the feed. To add a comment, switch to one of the following options.
    Please register or sign in to reply
    0 Assignees
    Assign to
    None
    Milestone
    None
    Assign milestone
    None
    Time tracking
    No estimate or time spent
    None
    Due date
    None
    0
    Labels
    None
    Assign labels
    • No matching results
    • Manage project labels
    Confidentiality
    Not confidential
    Lock issue
    Unlocked
    participants
    Reference: obitools/obitools3#135