Comments (11)
Ok so... New Year and I have finally taken enough of a break and reset to fix this issue once and for all and it's a stupid mistake XD.
So basically my issue was an indentation (or lack thereof) in my schema reference file. Basically, I messed up the indentation, so I had:
name: "My Pipeline"
description: My Description
imports:
- http://schema.databio.org/pep/2.1.0.yaml
properties:
samples:
type: array
items:
...<insert rest here>
when I needed to add indents like so:
name: "My Pipeline"
description: My Description
imports:
- http://schema.databio.org/pep/2.1.0.yaml
properties:
samples:
type: array
items:
...<insert rest here>
The moral of the story, if you can't fix it and you're getting frustrated, take a break 😂
from eido.
Could you post your PEP and the schema here? Also, what versions of peppy and eido are you using?
from eido.
Peppy: v0.31.1
Eido: v0.1.5
PEP:
description: "Schema for PEP sample declaration information for this workflow"
pep_version: "2.1.0"
sample_table: "samples.csv"
subsample_table: "partitions.csv"
**PEP Schema: **
name: "My Pipeline"
description: My Description
imports:
- http://schema.databio.org/pep/2.0.0.yaml
properties:
sample_name:
type: string
description: Identifier
dataset:
type: string
description: Dataset Name
file:
type: string
description: Filename
reference_genome:
type: string
description: reference ID
required:
- sample_name
- dataset
- file
- reference_genome
files:
- file
from eido.
I can confirm this also happens on eido v0.1.6-dev
from eido.
Hi @G-kodes your schema isn't reflecting the structure of the PEP correctly, which has both config
and samples
sections. Update your schema to this:
name: "My Pipeline"
description: My Description
imports:
- http://schema.databio.org/pep/2.0.0.yaml
properties:
samples:
type: array
items:
type: object
properties:
sample_name:
type: string
description: Identifier
dataset:
type: string
description: Dataset Name
file:
type: string
description: Filename
reference_genome:
type: string
description: reference ID
required:
- sample_name
- dataset
- file
- reference_genome
files:
- file
required:
- samples
It should work. You can see some more example schemas here: https://github.com/databio/schema.databio.org/tree/master/pipelines
from eido.
Your schema wasn't what you intended, but also, eido wasn't correctly interpreting this schema (which should have been asking for those properties on the main item). So, the error should have been different than what it showed, which may have pointed you down the path to fix your schema.
So, I've fixed the error message, so this will be more informative if a future user tries to do the same thing. This will show up on the next release (0.1.6).
from eido.
@nsheff thank you very much for the help! I guess this is chalked up to me not understanding the documentation properly so feeling a bit dumb this side XD.
I have updated my schema as you have directed, however unfortunately I am still getting an error:
Traceback (most recent call last):
File "/opt/homebrew/lib/python3.9/site-packages/snakemake/__init__.py", line 593, in snakemake
workflow.include(
File "/opt/homebrew/lib/python3.9/site-packages/snakemake/workflow.py", line 1182, in include
exec(compile(code, snakefile.get_path_or_uri(), "exec"), self.globals)
File "/Users/g-kodes/Documents/Pharmacogenetic-Analysis-Pipeline/workflow/Snakefile", line 37, in <module>
File "/opt/homebrew/lib/python3.9/site-packages/snakemake/workflow.py", line 1267, in pepschema
eido.validate_project(project=pep, schema=schema, exclude_case=True)
File "/opt/homebrew/lib/python3.9/site-packages/eido/validation.py", line 45, in validate_project
_validate_object(project_dict, preprocess_schema(schema_dict), exclude_case)
File "/opt/homebrew/lib/python3.9/site-packages/eido/schema.py", line 32, in preprocess_schema
"items" in schema_dict[PROP_KEY]["_samples"]
TypeError: argument of type 'NoneType' is not iterable
Am I just missing something basic here?
from eido.
Hmm. I am not reproducing this, using either eido 0.1.5 or 0.1.6. Can you provide your complete PEP (you posted the yaml, I need the csvs) by chance? This error appears to be indicating that you don't have any samples in your PEP, but that's confusing to me. It looks like the samples
array in your schema isn't being recognized.
Are you sure you copied the schema correctly?
from eido.
Also, you might try with the latest dev version which is almost released, (0.1.6)
pip install https://github.com/pepkit/eido/archive/refs/heads/dev.zip
You can check version with eido --version
.
from eido.
Hi! My apologies, this is turning into a support ticket!
My edit version output is eido 0.1.5
and my complete PEP file is:
description: "Schema for PEP sample declaration information for this workflow"
pep_version: "2.1.0"
sample_table: "samples.csv"
I have attempted to use the newer Eido version you indicated above, however it appears to be dependant on an un-released version of peppy, version >=0.32.0
. I can only find up till version 0.31.2
online?
from eido.
If samples.csv
doesn't exist, you should be getting this exception:
peppy.exceptions.SampleTableFileException: Could not read table: samples.csv. Caught exception: FileNotFoundError(2, 'No such file or directory')
If I do have a CSV file with at least 1 sample, I can validate that PEP with eido 0.1.5:
cat samples.csv
sample_name,attr,dataset,file,reference_genome
test1,test2,test,test,test
eido --version
> eido 0.1.5
eido validate pep.yaml -s pep_schema.yaml
> Validation successful
cat pep.yaml
Result:
description: "Schema for PEP sample declaration information for this workflow"
pep_version: "2.1.0"
sample_table: "samples.csv"
And for the schema:
cat pep_schema.yaml
Result:
name: "My Pipeline"
description: My Description
imports:
- http://schema.databio.org/pep/2.0.0.yaml
properties:
samples:
type: array
items:
type: object
properties:
sample_name:
type: string
description: Identifier
dataset:
type: string
description: Dataset Name
file:
type: string
description: Filename
reference_genome:
type: string
description: reference ID
required:
- sample_name
- dataset
- file
- reference_genome
files:
- file
required:
- samples
Really, eido 0.1.5
should be working for you. But anyway, if you want to upgrade, peppy 0.32.0 is pending release. You can install 0.32.0
of peppy in the same way, sorry about that:
pip install https://github.com/pepkit/eido/archive/refs/heads/dev.zip
from eido.
Related Issues (20)
- ImportError: cannot import name 'load_yaml' from 'peppy.utils' HOT 1
- Warn if bad filter HOT 2
- list of available output formats HOT 5
- CSV filter output for multi-value sample attributes HOT 1
- Uniform eido validation behavior for inputs HOT 1
- `convert_project` exits program HOT 1
- Processed PEP filter HOT 10
- Allow specifying `index_column_name` without `config.yaml` HOT 4
- CSV filter bug after peppy update HOT 1
- Upgrade the way errors are printed for CLI usage of eido
- Use eido with project modifiers HOT 1
- RemoteYAMLError HOT 6
- Unused parameter in validate_* functions HOT 2
- removelarge historical files HOT 1
- Object type for mutable mapping HOT 1
- Handling errors HOT 1
- Eido should use the same attribute names as pipestat for its schemas
- eido validate is broken in v0.2 HOT 1
- Hitting an `ImportError` through indirect dependency on `eido` on Python 3.10 HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from eido.