**************************
*****BioNano Genomics*****
******BNG-NGS Merge*******
**************************

ExportAGP.pl 11203 2020-06-18 21:28:10Z jwang

Running on host: 23d92ca2e664

Process ID (PID): 997
Start time: 2023-01-31 17:02:19

COMMAND
perl /home/bionano/tools/access/1.0/../../pipeline/1.0/HybridScaffold/1.0/hybridScaffold.pl -c hybridScaffold_DLE1_config_082522.xml -b EXP_REFINEFINAL1.cmap -n Calcutta4.hap1.hic.filtered.fasta -u CTTAAG -z cur_results_10236013_10236352.zip -w status_10236013_10236352.txt -M conflicts_cut_status_CTTAAG_10236013_10236352.txt -g -B 2 -N 2 -r /home/bionano/tools/access/1.0/../../pipeline/1.0/RefAligner/1.0/RefAligner -p /home/bionano/tools/access/1.0/../../pipeline/1.0/Pipeline/1.0 -o /home/bionano/access/local/jobs/10236352/output

NGS file: /home/bionano/access/local/jobs/10236352/output/fa2cmap/Calcutta4.hap1.hic.filtered.fasta 
BNG file: /home/bionano/access/local/jobs/10236352/EXP_REFINEFINAL1.cmap 
Configuration file: /home/bionano/access/local/jobs/10236352/hybridScaffold_DLE1_config_082522.xml 
Output folder: /home/bionano/access/local/jobs/10236352/output
User-defined conflict status file: /home/bionano/access/local/jobs/10236352/conflicts_cut_status_CTTAAG_10236013_10236352.txt
RefAligner binary: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner

Beginning cut_conflicts based on input a conflict file...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/cut_conflicts.pl -oriGMFile /home/bionano/access/local/jobs/10236352/output/align0/EXP_REFINEFINAL1_bppAdjust.cmap -oriSeqFile /home/bionano/access/local/jobs/10236352/output/fa2cmap/Calcutta4.hap1.hic.filtered_CTTAAG_0kb_0labels.cmap -outDir /home/bionano/access/local/jobs/10236352/output/assignAlignType/cut_conflicts_M1 -outGMFile /home/bionano/access/local/jobs/10236352/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -outSeqFile /home/bionano/access/local/jobs/10236352/output/assignAlignType/cut_conflicts_M1/Calcutta4.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -modBkptStatusFile /home/bionano/access/local/jobs/10236352/conflicts_cut_status_CTTAAG_10236013_10236352.txt -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
cut_conflicts complete in 3 seconds.
48 BNG contigs are found after the cut-conflict step
128 NGS contigs are found after the cut-conflict step

Beginning MergeNGS_BN...
*Using user-defined conflict-cut BioNano and sequence CMAP files*
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/MergeNGS_BN.pl -outputDir /home/bionano/access/local/jobs/10236352/output/mergeNGS_BN_M1 -refaligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -logFile mergeNGS_BN_script.log -ngs_cmap_fn /home/bionano/access/local/jobs/10236352/output/assignAlignType/cut_conflicts_M1/Calcutta4.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -bng_cmap_fn /home/bionano/access/local/jobs/10236352/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -id_shift 100000 -readparameters /home/bionano/access/local/jobs/10236352/output/align1/align1.errbin -xmlFile /home/bionano/access/local/jobs/10236352/hybridScaffold_DLE1_config_082522.xml
MergeNGS_BN complete in 21 seconds.

Beginning extraction of used and not used NGS contigs in hybrid scaffold pairmerge step...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/find_used_not_used_ngs.pl /home/bionano/access/local/jobs/10236352/output/mergeNGS_BN_M1 100000 /home/bionano/access/local/jobs/10236352/output/assignAlignType/cut_conflicts_M1/Calcutta4.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
Extraction of NGS contigs complete in 1 second.

Beginning alignment of NGS cmap to Hybrid CMAP...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/align_final_seq_two_passes.pl -outDir /home/bionano/access/local/jobs/10236352/output/align_final_M1 -outFilePrefix EXP_REFINEFINAL1_bppAdjust_cmap_Calcutta4_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -xmlFile /home/bionano/access/local/jobs/10236352/hybridScaffold_DLE1_config_082522.xml -seqFile /home/bionano/access/local/jobs/10236352/output/assignAlignType/cut_conflicts_M1/Calcutta4.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -scaffoldFile /home/bionano/access/local/jobs/10236352/output/align_final_M1/step2.hybrid.cmap -logFile /home/bionano/access/local/jobs/10236352/output/align_final_M1/align_final_seq_script_log.txt
align sequences to hybrid scaffolds completed  in 44 seconds.

Beginning extraction of used and not used BNG contigs in hybrid scaffold pairmerge step...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/find_used_not_used_bn.pl /home/bionano/access/local/jobs/10236352/output/mergeNGS_BN_M1 100000 /home/bionano/access/local/jobs/10236352/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
Extraction of BNG contigs complete in 1 second.

Beginning alignment of BNG cmap to Hybrid CMAP...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/align_final_bng.pl -outDir /home/bionano/access/local/jobs/10236352/output/align_final_M1 -outFilePrefix EXP_REFINEFINAL1_bppAdjust_cmap_Calcutta4_hap1_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -xmlFile /home/bionano/access/local/jobs/10236352/hybridScaffold_DLE1_config_082522.xml -bngFile /home/bionano/access/local/jobs/10236352/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -scaffoldFile /home/bionano/access/local/jobs/10236352/output/align_final_M1/step2.hybrid.cmap -logFile /home/bionano/access/local/jobs/10236352/output/align_final_M1/align_final_bng_log.txt
align Bionano genome maps to hybrid scaffolds completed  in 33 seconds.

Merging Hybrid CMAP with NGS not participated in the hybrid scaffold...
Running command: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -f -i /home/bionano/access/local/jobs/10236352/output/align_final_M1/step2.hybrid.cmap -i /home/bionano/access/local/jobs/10236352/output/mergeNGS_BN_M1/all_not_used_ngs.cmap -o HYBRID_SCAFFOLD_notUsedNGS_merged -merge -minsites 0 -stdout -stderr
Merging Hybrid CMAP with naive NGS CMAP complete in no time.

Merging Hybrid CMAP with BNG CMAP not participated in the hybrid scaffold...
Running command: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -f -i /home/bionano/access/local/jobs/10236352/output/align_final_M1/step2.hybrid.cmap -i /home/bionano/access/local/jobs/10236352/output/mergeNGS_BN_M1/all_not_used_bng.cmap -o HYBRID_SCAFFOLD_notUsedBNG_merged -merge -stdout -stderr
Merging Hybrid CMAP with naive BNG CMAP complete in 1 second.

Beginnning construction of AGP and FASTA file of the scaffolded and unscaffolded sequences...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/ExportAGP.pl -i /home/bionano/access/local/jobs/10236352/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Calcutta4_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap -c /home/bionano/access/local/jobs/10236352/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Calcutta4_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_r.cmap -o /home/bionano/access/local/jobs/10236352/output/agp_fasta_M1 -m /home/bionano/access/local/jobs/10236352/output/fa2cmap/Calcutta4.hap1.hic.filtered_CTTAAG_0kb_0labels_key.txt -s /home/bionano/access/local/jobs/10236352/output/fa2cmap/Calcutta4.hap1.hic.filtered.fasta -e CTTAAG 1 -r /home/bionano/access/local/jobs/10236352/output/assignAlignType/cut_conflicts_M1/auto_cut_NGS_coord_translation.txt -t
AGP and FASTA generation complete in 42 seconds.

Calculating statistics...

Original BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236352/EXP_REFINEFINAL1.cmap

Count  = 48
Min length (Mbp) = 0.130
Median length (Mbp) = 6.766
Mean length (Mbp) = 9.757
N50 length (Mbp) = 17.587
Max length (Mbp) = 37.969
Total length (Mbp) = 468.320

Bpp-adjusted BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236352/output/align0/EXP_REFINEFINAL1_bppAdjust.cmap

Count  = 48
Min length (Mbp) = 0.130
Median length (Mbp) = 6.766
Mean length (Mbp) = 9.757
N50 length (Mbp) = 17.588
Max length (Mbp) = 37.971
Total length (Mbp) = 468.338

Original NGS sequences statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236352/output/fa2cmap/Calcutta4.hap1.hic.filtered_CTTAAG_0kb_0labels.cmap

Count  = 132
Min length (Mbp) = 0.070
Median length (Mbp) = 0.381
Mean length (Mbp) = 3.674
N50 length (Mbp) = 12.208
Max length (Mbp) = 41.234
Total length (Mbp) = 484.920

Conflict Resolution from BNG-NGS alignment:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_conflicts_cut_stats.pl /home/bionano/access/local/jobs/10236352/output/assignAlignType/cut_conflicts_M1/conflicts_cut_status.txt

Number of conflict cuts made to Bionano maps: 0
Number of conflict cuts made to NGS sequences: 2
Number of Bionano maps to be cut: 0
Number of NGS sequences to be cut: 1

Before merge: BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236352/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap

Count  = 48
Min length (Mbp) = 0.130
Median length (Mbp) = 6.766
Mean length (Mbp) = 9.757
N50 length (Mbp) = 17.588
Max length (Mbp) = 37.971
Total length (Mbp) = 468.338

Before merge: NGS sequences statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236352/output/assignAlignType/cut_conflicts_M1/Calcutta4.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap

Count  = 128
Min length (Mbp) = 0.001
Median length (Mbp) = 0.398
Mean length (Mbp) = 3.781
N50 length (Mbp) = 12.208
Max length (Mbp) = 41.234
Total length (Mbp) = 484.017

BNG Genome Map in hybrid scaffold statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236352/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Calcutta4_hap1_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD_q.cmap

Count  = 45
Min length (Mbp) = 0.166
Median length (Mbp) = 7.379
Mean length (Mbp) = 10.379
N50 length (Mbp) = 17.588
Max length (Mbp) = 37.971
Total length (Mbp) = 467.052

NGS sequences in hybrid scaffold (CMAP) statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236352/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Calcutta4_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_q.cmap

Count  = 68
Min length (Mbp) = 0.089
Median length (Mbp) = 3.550
Mean length (Mbp) = 6.776
N50 length (Mbp) = 12.406
Max length (Mbp) = 41.234
Total length (Mbp) = 460.782

Hybrid scaffold Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236352/output/align_final_M1/step2.hybrid.cmap

Count  = 23
Min length (Mbp) = 1.661
Median length (Mbp) = 17.112
Mean length (Mbp) = 20.275
N50 length (Mbp) = 28.510
Max length (Mbp) = 47.249
Total length (Mbp) = 466.319

NGS FASTA sequence in hybrid scaffold statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_scaffolded_seq_fasta_stats.pl /home/bionano/access/local/jobs/10236352/output/agp_fasta_M1/Calcutta4.hap1.hic.filtered.fasta.cut.fasta /home/bionano/access/local/jobs/10236352/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Calcutta4_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_NOT_SCAFFOLDED.fasta

Count  = 68
Min length (Mbp) = 0.086
Median length (Mbp) = 3.548
Mean length (Mbp) = 6.774
N50 length (Mbp) = 12.406
Max length (Mbp) = 41.234
Total length (Mbp) = 460.661

Hybrid scaffold FASTA statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_fasta_stats.pl /home/bionano/access/local/jobs/10236352/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Calcutta4_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.fasta

Count  = 23
Min length (Mbp) = 1.604
Median length (Mbp) = 17.113
Mean length (Mbp) = 20.111
N50 length (Mbp) = 27.678
Max length (Mbp) = 47.109
Total length (Mbp) = 462.545

Hybrid scaffold FASTA plus not scaffolded NGS FASTA statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_hybrid_scaffold_not_scaffolded_seq_fasta_stats.pl /home/bionano/access/local/jobs/10236352/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Calcutta4_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.fasta /home/bionano/access/local/jobs/10236352/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Calcutta4_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_NOT_SCAFFOLDED.fasta

Count  = 100
Min length (Mbp) = 0.000
Median length (Mbp) = 0.168
Mean length (Mbp) = 4.868
N50 length (Mbp) = 27.678
Max length (Mbp) = 47.109
Total length (Mbp) = 486.803


Calculating CMAP statistics complete in 16 seconds.

Calculating XMAP statistics...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_xmap_stats.pl /home/bionano/access/local/jobs/10236352/output/align_final_M1 EXP_REFINEFINAL1_bppAdjust_cmap_Calcutta4_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap step2.hybrid.cmap EXP_REFINEFINAL1_bppAdjust_cmap_Calcutta4_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap.stats

refContig	refContig_size	query_cov_bp	query_unique_cov_bp	query_unique_cov_percent
8	14932226.2	14865087.6	14865087.6	99.55
100014	12261460.4	12225642.1	12225642.1	99.71
9	45106518.9	43529860.1	43498327.3	96.43
51	1660565.9	1651952	1651952	99.48
100036	1788056.8	1572682.8	1572682.8	87.95
5	17112253.1	17036322.5	17036322.5	99.56
47	2012885.6	2004086	2004086	99.56
23	27642166.4	27556594.1	27556594.1	99.69
7	17313673.8	17022653.1	17022653.1	98.32
100035	1954395.9	1928520.1	1928520.1	98.68
21	15716348.3	15647067.8	15647067.8	99.56
13	47249410.2	46821306.1	46800980	99.05
100005	28510071.4	27097363.3	27086588.2	95.01
2	26481977.6	26148916.7	26148916.7	98.74
49	8919845.6	7635488.4	7635488.4	85.60
10	44825961.9	44544224.8	44544224.8	99.37
100024	6785752.3	6779901.6	6779901.6	99.91
100017	8989233.9	8809983.3	8809983.3	98.01
4	28159539.5	27613664.8	27613664.8	98.06
11	19758348.4	19656600.6	19656600.6	99.49
1	41263717.4	41188849.7	41188849.7	99.82
32	45726557.1	45407013.6	45407013.6	99.30
46	2148450.4	2126551.7	2126551.7	98.98

Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_xmap_stats.pl /home/bionano/access/local/jobs/10236352/output/align_final_M1 EXP_REFINEFINAL1_bppAdjust_cmap_Calcutta4_hap1_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD.xmap step2.hybrid.cmap EXP_REFINEFINAL1_bppAdjust_cmap_Calcutta4_hap1_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD.xmap.stats

refContig	refContig_size	query_cov_bp	query_unique_cov_bp	query_unique_cov_percent
47	2012885.6	2006690.8	2006690.8	99.69
8	14932226.2	14882117.1	14882117.1	99.66
100035	1954395.9	1928520.1	1928520.1	98.68
1	41263717.4	41144597.2	41139937.2	99.70
49	8919845.6	8861649.3	8861649.3	99.35
9	45106518.9	45019812.4	45019812.4	99.81
21	15716348.3	15692395.3	15692395.3	99.85
5	17112253.1	17099525.8	17099525.8	99.93
11	19758348.4	19872042.9	19748210.4	99.95
7	17313673.8	17037977.3	17001040.8	98.19
32	45726557.1	45599888.1	45599888.1	99.72
100017	8989233.9	8929726	8929726	99.34
13	47249410.2	47337264.2	47162792.2	99.82
23	27642166.4	27740425.4	27608958.9	99.88
100036	1788056.8	1745697	1745697	97.63
100024	6785752.3	6779901.6	6779901.6	99.91
100014	12261460.4	12251924.6	12251924.6	99.92
2	26481977.6	26466015.2	26466015.2	99.94
100005	28510071.4	28458461.2	28458461.2	99.82
10	44825961.9	44753356.2	44744274.2	99.82
46	2148450.4	2126551.7	2126551.7	98.98
4	28159539.5	28147873.5	28120453.5	99.86
51	1660565.9	1646138	1646138	99.13

XMAP statistics calculation complete in no time.

Beginning archive and zip of result files...
