**************************
*****BioNano Genomics*****
******BNG-NGS Merge*******
**************************

ExportAGP.pl 11203 2020-06-18 21:28:10Z jwang

Running on host: a0fffe687d60

Process ID (PID): 998
Start time: 2023-01-31 00:52:15

COMMAND
perl /home/bionano/tools/access/1.0/../../pipeline/1.0/HybridScaffold/1.0/hybridScaffold.pl -c hybridScaffold_DLE1_config_082522.xml -b EXP_REFINEFINAL1.cmap -n Maleb.hap1.hic.filtered.fasta -u CTTAAG -z cur_results_10236347_10236349.zip -w status_10236347_10236349.txt -M conflicts_cut_status_CTTAAG_10236347_10236349.txt -g -B 2 -N 2 -r /home/bionano/tools/access/1.0/../../pipeline/1.0/RefAligner/1.0/RefAligner -p /home/bionano/tools/access/1.0/../../pipeline/1.0/Pipeline/1.0 -o /home/bionano/access/local/jobs/10236349/output

NGS file: /home/bionano/access/local/jobs/10236349/output/fa2cmap/Maleb.hap1.hic.filtered.fasta 
BNG file: /home/bionano/access/local/jobs/10236349/EXP_REFINEFINAL1.cmap 
Configuration file: /home/bionano/access/local/jobs/10236349/hybridScaffold_DLE1_config_082522.xml 
Output folder: /home/bionano/access/local/jobs/10236349/output
User-defined conflict status file: /home/bionano/access/local/jobs/10236349/conflicts_cut_status_CTTAAG_10236347_10236349.txt
RefAligner binary: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner

Beginning cut_conflicts based on input a conflict file...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/cut_conflicts.pl -oriGMFile /home/bionano/access/local/jobs/10236349/output/align0/EXP_REFINEFINAL1_bppAdjust.cmap -oriSeqFile /home/bionano/access/local/jobs/10236349/output/fa2cmap/Maleb.hap1.hic.filtered_CTTAAG_0kb_0labels.cmap -outDir /home/bionano/access/local/jobs/10236349/output/assignAlignType/cut_conflicts_M1 -outGMFile /home/bionano/access/local/jobs/10236349/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -outSeqFile /home/bionano/access/local/jobs/10236349/output/assignAlignType/cut_conflicts_M1/Maleb.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -modBkptStatusFile /home/bionano/access/local/jobs/10236349/conflicts_cut_status_CTTAAG_10236347_10236349.txt -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
cut_conflicts complete in 3 seconds.
22 BNG contigs are found after the cut-conflict step
136 NGS contigs are found after the cut-conflict step

Beginning MergeNGS_BN...
*Using user-defined conflict-cut BioNano and sequence CMAP files*
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/MergeNGS_BN.pl -outputDir /home/bionano/access/local/jobs/10236349/output/mergeNGS_BN_M1 -refaligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -logFile mergeNGS_BN_script.log -ngs_cmap_fn /home/bionano/access/local/jobs/10236349/output/assignAlignType/cut_conflicts_M1/Maleb.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -bng_cmap_fn /home/bionano/access/local/jobs/10236349/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -id_shift 100000 -readparameters /home/bionano/access/local/jobs/10236349/output/align1/align1.errbin -xmlFile /home/bionano/access/local/jobs/10236349/hybridScaffold_DLE1_config_082522.xml
MergeNGS_BN complete in 26 seconds.

Beginning extraction of used and not used NGS contigs in hybrid scaffold pairmerge step...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/find_used_not_used_ngs.pl /home/bionano/access/local/jobs/10236349/output/mergeNGS_BN_M1 100000 /home/bionano/access/local/jobs/10236349/output/assignAlignType/cut_conflicts_M1/Maleb.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
Extraction of NGS contigs complete in 1 second.

Beginning alignment of NGS cmap to Hybrid CMAP...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/align_final_seq_two_passes.pl -outDir /home/bionano/access/local/jobs/10236349/output/align_final_M1 -outFilePrefix EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -xmlFile /home/bionano/access/local/jobs/10236349/hybridScaffold_DLE1_config_082522.xml -seqFile /home/bionano/access/local/jobs/10236349/output/assignAlignType/cut_conflicts_M1/Maleb.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -scaffoldFile /home/bionano/access/local/jobs/10236349/output/align_final_M1/step2.hybrid.cmap -logFile /home/bionano/access/local/jobs/10236349/output/align_final_M1/align_final_seq_script_log.txt
align sequences to hybrid scaffolds completed  in 55 seconds.

Beginning extraction of used and not used BNG contigs in hybrid scaffold pairmerge step...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/find_used_not_used_bn.pl /home/bionano/access/local/jobs/10236349/output/mergeNGS_BN_M1 100000 /home/bionano/access/local/jobs/10236349/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
Extraction of BNG contigs complete in 2 seconds.

Beginning alignment of BNG cmap to Hybrid CMAP...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/align_final_bng.pl -outDir /home/bionano/access/local/jobs/10236349/output/align_final_M1 -outFilePrefix EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap1_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -xmlFile /home/bionano/access/local/jobs/10236349/hybridScaffold_DLE1_config_082522.xml -bngFile /home/bionano/access/local/jobs/10236349/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -scaffoldFile /home/bionano/access/local/jobs/10236349/output/align_final_M1/step2.hybrid.cmap -logFile /home/bionano/access/local/jobs/10236349/output/align_final_M1/align_final_bng_log.txt
align Bionano genome maps to hybrid scaffolds completed  in 42 seconds.

Merging Hybrid CMAP with NGS not participated in the hybrid scaffold...
Running command: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -f -i /home/bionano/access/local/jobs/10236349/output/align_final_M1/step2.hybrid.cmap -i /home/bionano/access/local/jobs/10236349/output/mergeNGS_BN_M1/all_not_used_ngs.cmap -o HYBRID_SCAFFOLD_notUsedNGS_merged -merge -minsites 0 -stdout -stderr
Merging Hybrid CMAP with naive NGS CMAP complete in no time.

Merging Hybrid CMAP with BNG CMAP not participated in the hybrid scaffold...
Running command: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -f -i /home/bionano/access/local/jobs/10236349/output/align_final_M1/step2.hybrid.cmap -i /home/bionano/access/local/jobs/10236349/output/mergeNGS_BN_M1/all_not_used_bng.cmap -o HYBRID_SCAFFOLD_notUsedBNG_merged -merge -stdout -stderr
Merging Hybrid CMAP with naive BNG CMAP complete in 1 second.

Beginnning construction of AGP and FASTA file of the scaffolded and unscaffolded sequences...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/ExportAGP.pl -i /home/bionano/access/local/jobs/10236349/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap -c /home/bionano/access/local/jobs/10236349/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_r.cmap -o /home/bionano/access/local/jobs/10236349/output/agp_fasta_M1 -m /home/bionano/access/local/jobs/10236349/output/fa2cmap/Maleb.hap1.hic.filtered_CTTAAG_0kb_0labels_key.txt -s /home/bionano/access/local/jobs/10236349/output/fa2cmap/Maleb.hap1.hic.filtered.fasta -e CTTAAG 1 -r /home/bionano/access/local/jobs/10236349/output/assignAlignType/cut_conflicts_M1/auto_cut_NGS_coord_translation.txt -t
AGP and FASTA generation complete in 45 seconds.

Calculating statistics...

Original BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236349/EXP_REFINEFINAL1.cmap

Count  = 26
Min length (Mbp) = 0.146
Median length (Mbp) = 19.148
Mean length (Mbp) = 19.014
N50 length (Mbp) = 31.143
Max length (Mbp) = 46.734
Total length (Mbp) = 494.356

Bpp-adjusted BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236349/output/align0/EXP_REFINEFINAL1_bppAdjust.cmap

Count  = 26
Min length (Mbp) = 0.146
Median length (Mbp) = 19.144
Mean length (Mbp) = 19.010
N50 length (Mbp) = 31.137
Max length (Mbp) = 46.725
Total length (Mbp) = 494.262

Original NGS sequences statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236349/output/fa2cmap/Maleb.hap1.hic.filtered_CTTAAG_0kb_0labels.cmap

Count  = 140
Min length (Mbp) = 0.072
Median length (Mbp) = 0.375
Mean length (Mbp) = 3.633
N50 length (Mbp) = 20.132
Max length (Mbp) = 50.903
Total length (Mbp) = 508.564

Conflict Resolution from BNG-NGS alignment:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_conflicts_cut_stats.pl /home/bionano/access/local/jobs/10236349/output/assignAlignType/cut_conflicts_M1/conflicts_cut_status.txt

Number of conflict cuts made to Bionano maps: 0
Number of conflict cuts made to NGS sequences: 3
Number of Bionano maps to be cut: 0
Number of NGS sequences to be cut: 3

Before merge: BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236349/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap

Count  = 22
Min length (Mbp) = 0.146
Median length (Mbp) = 20.659
Mean length (Mbp) = 22.325
N50 length (Mbp) = 31.137
Max length (Mbp) = 46.725
Total length (Mbp) = 491.146

Before merge: NGS sequences statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236349/output/assignAlignType/cut_conflicts_M1/Maleb.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap

Count  = 136
Min length (Mbp) = 0.072
Median length (Mbp) = 0.407
Mean length (Mbp) = 3.725
N50 length (Mbp) = 19.303
Max length (Mbp) = 50.903
Total length (Mbp) = 506.651

BNG Genome Map in hybrid scaffold statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236349/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap1_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD_q.cmap

Count  = 19
Min length (Mbp) = 0.146
Median length (Mbp) = 27.981
Mean length (Mbp) = 25.715
N50 length (Mbp) = 31.137
Max length (Mbp) = 46.725
Total length (Mbp) = 488.588

NGS sequences in hybrid scaffold (CMAP) statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236349/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_q.cmap

Count  = 73
Min length (Mbp) = 0.103
Median length (Mbp) = 1.768
Mean length (Mbp) = 6.732
N50 length (Mbp) = 19.303
Max length (Mbp) = 50.903
Total length (Mbp) = 491.443

Hybrid scaffold Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236349/output/align_final_M1/step2.hybrid.cmap

Count  = 16
Min length (Mbp) = 0.111
Median length (Mbp) = 35.416
Mean length (Mbp) = 30.909
N50 length (Mbp) = 40.581
Max length (Mbp) = 54.545
Total length (Mbp) = 494.539

NGS FASTA sequence in hybrid scaffold statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_scaffolded_seq_fasta_stats.pl /home/bionano/access/local/jobs/10236349/output/agp_fasta_M1/Maleb.hap1.hic.filtered.fasta.cut.fasta /home/bionano/access/local/jobs/10236349/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_NOT_SCAFFOLDED.fasta

Count  = 72
Min length (Mbp) = 0.003
Median length (Mbp) = 1.813
Mean length (Mbp) = 6.804
N50 length (Mbp) = 19.281
Max length (Mbp) = 50.903
Total length (Mbp) = 489.867

Hybrid scaffold FASTA statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_fasta_stats.pl /home/bionano/access/local/jobs/10236349/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.fasta

Count  = 16
Min length (Mbp) = 0.111
Median length (Mbp) = 34.973
Mean length (Mbp) = 30.770
N50 length (Mbp) = 40.475
Max length (Mbp) = 54.063
Total length (Mbp) = 492.315

Hybrid scaffold FASTA plus not scaffolded NGS FASTA statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_hybrid_scaffold_not_scaffolded_seq_fasta_stats.pl /home/bionano/access/local/jobs/10236349/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.fasta /home/bionano/access/local/jobs/10236349/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_NOT_SCAFFOLDED.fasta

Count  = 114
Min length (Mbp) = 0.000
Median length (Mbp) = 0.144
Mean length (Mbp) = 4.483
N50 length (Mbp) = 40.475
Max length (Mbp) = 54.063
Total length (Mbp) = 511.012


Calculating CMAP statistics complete in 17 seconds.

Calculating XMAP statistics...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_xmap_stats.pl /home/bionano/access/local/jobs/10236349/output/align_final_M1 EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap step2.hybrid.cmap EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap.stats

refContig	refContig_size	query_cov_bp	query_unique_cov_bp	query_unique_cov_percent
58	530757.0	519719	519719	97.92
100003	40580916.7	40475074	40475074	99.74
34	15432108.2	15381337.3	15375269.9	99.63
8	36323960.4	36166250.8	36155034.2	99.53
50	21067423.0	21021395.6	21021395.6	99.78
56	596014.0	833539	588251	98.70
100005	34508031.0	33218386.9	33202502.9	96.22
100004	39564179.5	39321890.3	39176346.4	99.02
3	28054483.0	28003957.6	28003957.6	99.82
10	54544709.4	53221195.4	52608138.1	96.45
5	44793947.5	44597032.5	44597032.5	99.56
100001	46724970.0	45897736.2	45897736.2	98.23
1	50933353.0	50833322.3	50833322.3	99.80
4	47017118.2	46839600.7	46825924.9	99.59
120	110801.0	167317	107768	97.26
100006	33755807.8	33597106.8	33580302.6	99.48

Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_xmap_stats.pl /home/bionano/access/local/jobs/10236349/output/align_final_M1 EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap1_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD.xmap step2.hybrid.cmap EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap1_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD.xmap.stats

refContig	refContig_size	query_cov_bp	query_unique_cov_bp	query_unique_cov_percent
8	36323960.4	36615161.4	36278685.4	99.88
56	596014.0	0	0	0.00
100005	34508031.0	34449507.9	34449507.9	99.83
100006	33755807.8	33744030.6	33744030.6	99.97
1	50933353.0	51232707.9	50902871.9	99.94
120	110801.0	0	0	0.00
5	44793947.5	44775157	44775157	99.96
34	15432108.2	15188679.5	15188679.5	98.42
4	47017118.2	46628290.4	46606020.4	99.13
100003	40580916.7	40569058.1	40569058.1	99.97
50	21067423.0	21047773.8	21047773.8	99.91
100004	39564179.5	39553900.8	39553900.8	99.97
3	28054483.0	28017281.9	28017281.9	99.87
10	54544709.4	49455476.3	49455476.3	90.67
58	530757.0	0	0	0.00
100001	46724970.0	46710054.7	46710054.7	99.97

XMAP statistics calculation complete in no time.

Beginning archive and zip of result files...
