Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mimalloc: warning: unable to allocate aligned OS memory directly, fall back to over-allocation #1384

Open
1 task done
ranweiwei66 opened this issue Sep 25, 2024 · 1 comment
Labels

Comments

@ranweiwei66
Copy link

Description of bug

If I ignore this warning "mimalloc: warning: unable to allocate aligned OS memory directly, fall back to over-allocation (16538140672 bytes, address: 0x7f31c5449000, alignment: 67108864, commit: 1)"
"mimalloc: warning: unable to allocate aligned OS memory directly, fall back to over-allocation (16538140672 bytes, address: 0x7f2dea400000, alignment: 67108864, commit: 1)" and proceed with the results generated by it for further analysis, is that okay? thanks for your help very much.

spades.log

======= SPAdes pipeline started. Log can be found here: /home/sd01/RWW1/rww/rawdata/GX5/fastp/normalization15X/spades4/spades.log

/home/sd01/RWW1/rww/rawdata/GX5/fastp/normalization15X/1.nor.fq.gz: max reads length: 150
/home/sd01/RWW1/rww/rawdata/GX5/fastp/normalization15X/2.nor.fq.gz: max reads length: 150

Reads length: 150

Default k-mer sizes were set to [21, 33, 55, 77] because estimated read length (150) is equal to or greater than 150

===== Before start started.

===== Read error correction started.

===== Read error correction started.

== Running: /home/sd01/RWW1/rww/install/anaconda3/envs/spades4.0/bin/spades-hammer /home/sd01/RWW1/rww/rawdata/GX5/fastp/normalization15X/spades4/corrected/configs/config.info

0:00:00.000 1M / 9M INFO General (main.cpp : 76) Starting BayesHammer, built from N/A, git revision N/A
0:00:00.002 1M / 9M INFO General (main.cpp : 77) Loading config from "/home/sd01/RWW1/rww/rawdata/GX5/fastp/normalization15X/spades4/corrected/configs/config.info"
0:00:00.002 1M / 9M INFO General (main.cpp : 79) Maximum # of threads to use (adjusted due to OMP capabilities): 32
0:00:00.003 1M / 9M INFO General (memory_limit.cpp : 55) Memory limit set to 250 Gb
0:00:00.003 1M / 9M INFO General (main.cpp : 87) Trying to determine PHRED offset
0:00:00.005 1M / 9M INFO General (main.cpp : 93) Determined value is 33
0:00:00.005 1M / 9M INFO General (hammer_tools.cpp : 40) Hamming graph threshold tau=1, k=21, subkmer positions = [ 0 10 ]
0:00:00.005 1M / 9M INFO General (main.cpp : 114) Size of aux. kmer data 24 bytes
=== ITERATION 0 begins ===
0:00:00.005 1M / 9M INFO General (kmer_index_builder.hpp : 258) Splitting kmer instances into 16 files using 32 threads. This might take a while.
0:00:00.006 1M / 9M INFO General (file_limit.hpp : 43) Open file limit set to 1024
0:00:00.006 1M / 9M INFO General (kmer_splitter.hpp : 94) Memory available for splitting buffers: 2.60417 Gb
0:00:00.006 1M / 9M INFO General (kmer_splitter.hpp : 102) Using cell size of 4194304
0:00:00.013 19G / 19G INFO K-mer Splitting (kmer_data.cpp : 98) Processing "/home/sd01/RWW1/rww/rawdata/GX5/fastp/normalization15X/1.nor.fq.gz"
0:00:44.175 19G / 30G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 7636984 reads
0:01:29.238 19G / 31G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 15457007 reads
0:02:13.103 19G / 31G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 23046290 reads
0:02:58.138 19G / 31G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 30821559 reads
0:03:41.824 19G / 31G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 38475082 reads
0:04:12.364 19G / 31G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 43761961 reads
0:04:12.367 19G / 31G INFO K-mer Splitting (kmer_data.cpp : 98) Processing "/home/sd01/RWW1/rww/rawdata/GX5/fastp/normalization15X/2.nor.fq.gz"
0:04:56.643 19G / 31G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 51539163 reads
0:05:39.375 19G / 31G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 59012264 reads
0:06:23.214 19G / 31G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 66791118 reads
0:07:06.620 19G / 31G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 74421289 reads
0:07:50.553 19G / 31G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 82201049 reads
0:08:21.233 19G / 31G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 87523922 reads
0:08:21.238 19G / 31G INFO K-mer Splitting (kmer_data.cpp : 113) Total 87523922 reads processed
0:08:22.733 1M / 31G INFO General (kmer_index_builder.hpp : 264) Starting k-mer counting.
0:08:55.654 1M / 92G INFO General (kmer_index_builder.hpp : 275) K-mer counting done. There are 2066303430 kmers in total.
0:08:55.660 1M / 92G INFO K-mer Index Building (kmer_index_builder.hpp : 410) Building perfect hash indices
0:10:21.244 1475M / 92G INFO K-mer Index Building (kmer_index_builder.hpp : 446) Index built. Total 2066303430 kmers, 1492428856 bytes occupied (5.77816 bits per kmer).
0:10:21.247 1475M / 92G INFO K-mer Counting (kmer_data.cpp : 355) Arranging kmers in hash map order
mimalloc: warning: unable to allocate aligned OS memory directly, fall back to over-allocation (16538140672 bytes, address: 0x7f31c5449000, alignment: 67108864, commit: 1)
mimalloc: warning: unable to allocate aligned OS memory directly, fall back to over-allocation (16538140672 bytes, address: 0x7f2dea400000, alignment: 67108864, commit: 1)
0:11:36.469 33G / 92G INFO General (main.cpp : 149) Clustering Hamming graph.

params.txt

(spades4.0) sd01@sd01:~/RWW1/rww/rawdata/GX5/fastp/normalization15X$ time spades.py -1 1.nor.fq.gz -2 2.nor.fq.gz -o spades4 -t 32

SPAdes version

4.0.0

Operating System

Linux-5.15.0-117-generic-x86_64-with-glibc2.35

Python Version

3.8.19

Method of SPAdes installation

conda

No errors reported in spades.log

  • Yes
@asl
Copy link
Member

asl commented Sep 25, 2024

It is ok to ignore the warning.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants