hello,
I am trying to use misha/shaman for the first time with my micro C data. I managed to upload my track in a test and use shaman_shuffle_and_score_hic_mat() to score a small test region. Therefore my installation should be OK.
Then I created a track with my entire sample (which is pretty big ca. 85 Gb valid_interaction.txt). the upload took a while but it worked too. I started the shuffling with shaman_shuffle_hic_track() and it started running without any problem. it finished almost all chromosomes but after ~30h he gave me this error:
39 :: 147762666 35846685 0.24 9
39 :: 164178891 39829650 0.24 10
run time = 8.79229 hours
Error in do.ply(i) : task 2 failed - "std::bad_alloc"
Calls: shaman_shuffle_hic_track ... -> ldply -> llply -> ->
Execution halted
I used 20 cores and 450Gb of memory... was that not enough? could you please tell me what I could change to keep it running? thank you.
here is the script of how I run it:
R.script:
library(misha)
library(shaman)
options(shaman.mc_support=1)
mDBloc <- '/micro-c/misha/'
db <- 'mm10'
dbDir <- paste0(mDBloc,db,'/')
gsetroot(dbDir)
gdb.reload()
#gtrack.2d.import(track="mC_obs_sample2", description="observed_mC_data", file=c("valid_interaction.txt"))
#gdb.reload()
#Create shuffled track
currentTrack <- c('mC_obs_sample2')
outName <- 'hic.microC.sample2_shuffle'
currentWD <- '/work/cavalli/mszalay/micro-c/misha/'
#grdSmall <- 500
#grdHigh <- 1000
shaman_shuffle_hic_track(track_db=dbDir, obs_track_nm=currentTrack, exp_track_nm=outName, work_dir=currentWD, max_jobs=20)
print("end")
hello,
I am trying to use misha/shaman for the first time with my micro C data. I managed to upload my track in a test and use shaman_shuffle_and_score_hic_mat() to score a small test region. Therefore my installation should be OK.
Then I created a track with my entire sample (which is pretty big ca. 85 Gb valid_interaction.txt). the upload took a while but it worked too. I started the shuffling with shaman_shuffle_hic_track() and it started running without any problem. it finished almost all chromosomes but after ~30h he gave me this error:
39 :: 147762666 35846685 0.24 9
39 :: 164178891 39829650 0.24 10
run time = 8.79229 hours
Error in do.ply(i) : task 2 failed - "std::bad_alloc"
Calls: shaman_shuffle_hic_track ... -> ldply -> llply -> ->
Execution halted
I used 20 cores and 450Gb of memory... was that not enough? could you please tell me what I could change to keep it running? thank you.
here is the script of how I run it:
R.script:
library(misha)
library(shaman)
options(shaman.mc_support=1)
mDBloc <- '/micro-c/misha/'
db <- 'mm10'
dbDir <- paste0(mDBloc,db,'/')
gsetroot(dbDir)
gdb.reload()
#gtrack.2d.import(track="mC_obs_sample2", description="observed_mC_data", file=c("valid_interaction.txt"))
#gdb.reload()
#Create shuffled track
currentTrack <- c('mC_obs_sample2')
outName <- 'hic.microC.sample2_shuffle'
currentWD <- '/work/cavalli/mszalay/micro-c/misha/'
#grdSmall <- 500
#grdHigh <- 1000
shaman_shuffle_hic_track(track_db=dbDir, obs_track_nm=currentTrack, exp_track_nm=outName, work_dir=currentWD, max_jobs=20)
print("end")