How specific sequences are extracted for pantas analysis

[ld9866]

Hello developer:

As I was reading the literature, I noticed that you also encountered a situation where VG speeds were slow, which prevented us from doing large-scale analyses of mammalian pan-genomes.

However, it is feasible to extract a part of the sequence, such as the extraction of chromosome 2, 80-82Mb, and only analyze the candidate genes we are concerned about. I want to ask how you solve the annotation problem because the modification of the annotation position is confusing.

Best regards,
Dong

[ldenti]

Hi,
we did not extract regions but we restricted the analysis to a set of genes. In our experience, the real bottleneck is GCSA2 index construction. All previous steps to produce the graph are not that expensive. So we did not create a local graph or "shift" the annotation since it's always hard to reconcile, but we built the full chromosome graph and then simplified it as much as possible while retaining the information we needed.

If you have an annotation with the genes you are interested in (maybe those in the region you mentioned), you could try to: build the chromosome spliced pangenome using the small annotation (check out our snakemake) and then use the reduce.py script to remove all intergenic regions from the graph (it should create n connected components with n<=#genes).

If you don't have the annotation, it should be possible to create the chromosome graph (vg constuct > vg rna > vg gbwt > vg rna as done in our snakemake) and then moving on the reference path up to the regions you are interested in and extract the corresponding subgraph. This is the only idea that comes to my mind right now.

If you need more info or want to discuss more, let me know.

Best,

[ld9866]

What you said is normal. When VG builds the index, we have 27 genomes and 4 million variants, and using 2TB of running memory to build the index will still crash.
When aligning 6G of transcriptomes, it still takes more than 2 weeks on a 128-core server, which is impractical for large-scale analysis.
So I have now split the chromosomes, first aligned the transcriptome to the linear reference genome, extracted the sequence of each chromosome and aligned it to the single chromosome of the pan-genome. Now it seems to be fine and the speed is very fast.
I mentioned the issue of docker hosting in another question. Can you make a ready-made docker file so that we can just pull to reproduce your environment?
Best regards,
Dong