**************************
*****BioNano Genomics*****
******BNG-NGS Merge*******
**************************

ExportAGP.pl 11203 2020-06-18 21:28:10Z jwang

Running on host: 703800678e0b

Process ID (PID): 997
Start time: 2023-01-31 19:27:07

COMMAND
perl /home/bionano/tools/access/1.0/../../pipeline/1.0/HybridScaffold/1.0/hybridScaffold.pl -c hybridScaffold_DLE1_config_082522.xml -b EXP_REFINEFINAL1.cmap -n 9128-3.hap1.hic.filtered.fasta -u CTTAAG -z cur_results_10236355_10236356.zip -w status_10236355_10236356.txt -M conflicts_cut_status_CTTAAG_10236355_10236356.txt -g -B 2 -N 2 -r /home/bionano/tools/access/1.0/../../pipeline/1.0/RefAligner/1.0/RefAligner -p /home/bionano/tools/access/1.0/../../pipeline/1.0/Pipeline/1.0 -o /home/bionano/access/local/jobs/10236356/output

NGS file: /home/bionano/access/local/jobs/10236356/output/fa2cmap/9128-3.hap1.hic.filtered.fasta 
BNG file: /home/bionano/access/local/jobs/10236356/EXP_REFINEFINAL1.cmap 
Configuration file: /home/bionano/access/local/jobs/10236356/hybridScaffold_DLE1_config_082522.xml 
Output folder: /home/bionano/access/local/jobs/10236356/output
User-defined conflict status file: /home/bionano/access/local/jobs/10236356/conflicts_cut_status_CTTAAG_10236355_10236356.txt
RefAligner binary: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner

Beginning cut_conflicts based on input a conflict file...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/cut_conflicts.pl -oriGMFile /home/bionano/access/local/jobs/10236356/output/align0/EXP_REFINEFINAL1_bppAdjust.cmap -oriSeqFile /home/bionano/access/local/jobs/10236356/output/fa2cmap/9128-3.hap1.hic.filtered_CTTAAG_0kb_0labels.cmap -outDir /home/bionano/access/local/jobs/10236356/output/assignAlignType/cut_conflicts_M1 -outGMFile /home/bionano/access/local/jobs/10236356/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -outSeqFile /home/bionano/access/local/jobs/10236356/output/assignAlignType/cut_conflicts_M1/9128-3.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -modBkptStatusFile /home/bionano/access/local/jobs/10236356/conflicts_cut_status_CTTAAG_10236355_10236356.txt -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
cut_conflicts complete in 4 seconds.
27 BNG contigs are found after the cut-conflict step
100 NGS contigs are found after the cut-conflict step

Beginning MergeNGS_BN...
*Using user-defined conflict-cut BioNano and sequence CMAP files*
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/MergeNGS_BN.pl -outputDir /home/bionano/access/local/jobs/10236356/output/mergeNGS_BN_M1 -refaligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -logFile mergeNGS_BN_script.log -ngs_cmap_fn /home/bionano/access/local/jobs/10236356/output/assignAlignType/cut_conflicts_M1/9128-3.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -bng_cmap_fn /home/bionano/access/local/jobs/10236356/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -id_shift 100000 -readparameters /home/bionano/access/local/jobs/10236356/output/align1/align1.errbin -xmlFile /home/bionano/access/local/jobs/10236356/hybridScaffold_DLE1_config_082522.xml
MergeNGS_BN complete in 22 seconds.

Beginning extraction of used and not used NGS contigs in hybrid scaffold pairmerge step...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/find_used_not_used_ngs.pl /home/bionano/access/local/jobs/10236356/output/mergeNGS_BN_M1 100000 /home/bionano/access/local/jobs/10236356/output/assignAlignType/cut_conflicts_M1/9128-3.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
Extraction of NGS contigs complete in 1 second.

Beginning alignment of NGS cmap to Hybrid CMAP...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/align_final_seq_two_passes.pl -outDir /home/bionano/access/local/jobs/10236356/output/align_final_M1 -outFilePrefix EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -xmlFile /home/bionano/access/local/jobs/10236356/hybridScaffold_DLE1_config_082522.xml -seqFile /home/bionano/access/local/jobs/10236356/output/assignAlignType/cut_conflicts_M1/9128-3.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -scaffoldFile /home/bionano/access/local/jobs/10236356/output/align_final_M1/step2.hybrid.cmap -logFile /home/bionano/access/local/jobs/10236356/output/align_final_M1/align_final_seq_script_log.txt
align sequences to hybrid scaffolds completed  in 1 minute and 9 seconds.

Beginning extraction of used and not used BNG contigs in hybrid scaffold pairmerge step...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/find_used_not_used_bn.pl /home/bionano/access/local/jobs/10236356/output/mergeNGS_BN_M1 100000 /home/bionano/access/local/jobs/10236356/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
Extraction of BNG contigs complete in 1 second.

Beginning alignment of BNG cmap to Hybrid CMAP...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/align_final_bng.pl -outDir /home/bionano/access/local/jobs/10236356/output/align_final_M1 -outFilePrefix EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap1_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -xmlFile /home/bionano/access/local/jobs/10236356/hybridScaffold_DLE1_config_082522.xml -bngFile /home/bionano/access/local/jobs/10236356/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -scaffoldFile /home/bionano/access/local/jobs/10236356/output/align_final_M1/step2.hybrid.cmap -logFile /home/bionano/access/local/jobs/10236356/output/align_final_M1/align_final_bng_log.txt
align Bionano genome maps to hybrid scaffolds completed  in 44 seconds.

Merging Hybrid CMAP with NGS not participated in the hybrid scaffold...
Running command: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -f -i /home/bionano/access/local/jobs/10236356/output/align_final_M1/step2.hybrid.cmap -i /home/bionano/access/local/jobs/10236356/output/mergeNGS_BN_M1/all_not_used_ngs.cmap -o HYBRID_SCAFFOLD_notUsedNGS_merged -merge -minsites 0 -stdout -stderr
Merging Hybrid CMAP with naive NGS CMAP complete in 1 second.

Merging Hybrid CMAP with BNG CMAP not participated in the hybrid scaffold...
Running command: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -f -i /home/bionano/access/local/jobs/10236356/output/align_final_M1/step2.hybrid.cmap -i /home/bionano/access/local/jobs/10236356/output/mergeNGS_BN_M1/all_not_used_bng.cmap -o HYBRID_SCAFFOLD_notUsedBNG_merged -merge -stdout -stderr
Merging Hybrid CMAP with naive BNG CMAP complete in no time.

Beginnning construction of AGP and FASTA file of the scaffolded and unscaffolded sequences...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/ExportAGP.pl -i /home/bionano/access/local/jobs/10236356/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap -c /home/bionano/access/local/jobs/10236356/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_r.cmap -o /home/bionano/access/local/jobs/10236356/output/agp_fasta_M1 -m /home/bionano/access/local/jobs/10236356/output/fa2cmap/9128-3.hap1.hic.filtered_CTTAAG_0kb_0labels_key.txt -s /home/bionano/access/local/jobs/10236356/output/fa2cmap/9128-3.hap1.hic.filtered.fasta -e CTTAAG 1 -r /home/bionano/access/local/jobs/10236356/output/assignAlignType/cut_conflicts_M1/auto_cut_NGS_coord_translation.txt -t
AGP and FASTA generation complete in 43 seconds.

Calculating statistics...

Original BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236356/EXP_REFINEFINAL1.cmap

Count  = 27
Min length (Mbp) = 0.332
Median length (Mbp) = 13.210
Mean length (Mbp) = 17.846
N50 length (Mbp) = 35.379
Max length (Mbp) = 49.369
Total length (Mbp) = 481.840

Bpp-adjusted BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236356/output/align0/EXP_REFINEFINAL1_bppAdjust.cmap

Count  = 27
Min length (Mbp) = 0.332
Median length (Mbp) = 13.206
Mean length (Mbp) = 17.841
N50 length (Mbp) = 35.369
Max length (Mbp) = 49.354
Total length (Mbp) = 481.698

Original NGS sequences statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236356/output/fa2cmap/9128-3.hap1.hic.filtered_CTTAAG_0kb_0labels.cmap

Count  = 102
Min length (Mbp) = 0.070
Median length (Mbp) = 0.261
Mean length (Mbp) = 4.892
N50 length (Mbp) = 21.604
Max length (Mbp) = 40.225
Total length (Mbp) = 499.018

Conflict Resolution from BNG-NGS alignment:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_conflicts_cut_stats.pl /home/bionano/access/local/jobs/10236356/output/assignAlignType/cut_conflicts_M1/conflicts_cut_status.txt

Number of conflict cuts made to Bionano maps: 0
Number of conflict cuts made to NGS sequences: 1
Number of Bionano maps to be cut: 0
Number of NGS sequences to be cut: 1

Before merge: BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236356/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap

Count  = 27
Min length (Mbp) = 0.332
Median length (Mbp) = 13.206
Mean length (Mbp) = 17.841
N50 length (Mbp) = 35.369
Max length (Mbp) = 49.354
Total length (Mbp) = 481.698

Before merge: NGS sequences statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236356/output/assignAlignType/cut_conflicts_M1/9128-3.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap

Count  = 100
Min length (Mbp) = 0.070
Median length (Mbp) = 0.261
Mean length (Mbp) = 4.973
N50 length (Mbp) = 21.604
Max length (Mbp) = 40.225
Total length (Mbp) = 497.332

BNG Genome Map in hybrid scaffold statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236356/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap1_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD_q.cmap

Count  = 26
Min length (Mbp) = 0.332
Median length (Mbp) = 13.688
Mean length (Mbp) = 18.511
N50 length (Mbp) = 35.369
Max length (Mbp) = 49.354
Total length (Mbp) = 481.286

NGS sequences in hybrid scaffold (CMAP) statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236356/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_q.cmap

Count  = 34
Min length (Mbp) = 0.129
Median length (Mbp) = 11.859
Mean length (Mbp) = 14.113
N50 length (Mbp) = 21.604
Max length (Mbp) = 40.225
Total length (Mbp) = 479.855

Hybrid scaffold Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236356/output/align_final_M1/step2.hybrid.cmap

Count  = 14
Min length (Mbp) = 2.853
Median length (Mbp) = 35.619
Mean length (Mbp) = 34.383
N50 length (Mbp) = 47.506
Max length (Mbp) = 56.520
Total length (Mbp) = 481.365

NGS FASTA sequence in hybrid scaffold statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_scaffolded_seq_fasta_stats.pl /home/bionano/access/local/jobs/10236356/output/agp_fasta_M1/9128-3.hap1.hic.filtered.fasta.cut.fasta /home/bionano/access/local/jobs/10236356/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_NOT_SCAFFOLDED.fasta

Count  = 34
Min length (Mbp) = 0.112
Median length (Mbp) = 11.855
Mean length (Mbp) = 14.108
N50 length (Mbp) = 21.604
Max length (Mbp) = 40.225
Total length (Mbp) = 479.671

Hybrid scaffold FASTA statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_fasta_stats.pl /home/bionano/access/local/jobs/10236356/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.fasta

Count  = 14
Min length (Mbp) = 2.662
Median length (Mbp) = 35.496
Mean length (Mbp) = 34.267
N50 length (Mbp) = 47.449
Max length (Mbp) = 56.316
Total length (Mbp) = 479.740

Hybrid scaffold FASTA plus not scaffolded NGS FASTA statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_hybrid_scaffold_not_scaffolded_seq_fasta_stats.pl /home/bionano/access/local/jobs/10236356/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.fasta /home/bionano/access/local/jobs/10236356/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_NOT_SCAFFOLDED.fasta

Count  = 96
Min length (Mbp) = 0.004
Median length (Mbp) = 0.152
Mean length (Mbp) = 5.199
N50 length (Mbp) = 47.449
Max length (Mbp) = 56.316
Total length (Mbp) = 499.087


Calculating CMAP statistics complete in 17 seconds.

Calculating XMAP statistics...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_xmap_stats.pl /home/bionano/access/local/jobs/10236356/output/align_final_M1 EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap step2.hybrid.cmap EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap.stats

refContig	refContig_size	query_cov_bp	query_unique_cov_bp	query_unique_cov_percent
100010	23476896.4	23306850.4	23306850.4	99.28
2	56520211.4	56267716.8	56258052.6	99.54
100002	47707593.9	47569791.9	47561580.3	99.69
100001	49354260.0	49076947.9	49070509.8	99.43
21	10582709.1	10522354.6	10522354.6	99.43
16	46516135.1	46428437.1	46409213	99.77
14	35506102.6	35473457.2	35462108.9	99.88
19	19425284.6	19308952.6	19308952.6	99.40
3	47505865.7	47390213	47375779.7	99.73
100005	35732767.3	35573668	35573668	99.55
4	29378138.2	29260031.8	29242268.3	99.54
100022	2853165.7	2654218	2654218	93.03
100009	25005321.2	24850254.6	24832471.3	99.31
1	51800346.0	51674607.6	51674607.6	99.76

Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_xmap_stats.pl /home/bionano/access/local/jobs/10236356/output/align_final_M1 EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap1_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD.xmap step2.hybrid.cmap EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap1_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD.xmap.stats

refContig	refContig_size	query_cov_bp	query_unique_cov_bp	query_unique_cov_percent
100010	23476896.4	23306850.4	23306850.4	99.28
100005	35732767.3	35606687.7	35606687.7	99.65
100022	2853165.7	2756939.3	2756939.3	96.63
100002	47707593.9	47589830.1	47589830.1	99.75
14	35506102.6	35336213.9	35336213.9	99.52
1	51800346.0	51691435.2	51684395.2	99.78
100001	49354260.0	49215131.4	49215131.4	99.72
4	29378138.2	29268943.7	29255866.7	99.58
100009	25005321.2	24976182.1	24976182.1	99.88
3	47505865.7	47360396.7	47360396.7	99.69
19	19425284.6	19041733.8	19018871.8	97.91
21	10582709.1	10507114.6	10507114.6	99.29
16	46516135.1	46477843.1	46477843.1	99.92
2	56520211.4	56568796.7	56405119.7	99.80

XMAP statistics calculation complete in no time.

Beginning archive and zip of result files...
