**************************
*****BioNano Genomics*****
******BNG-NGS Merge*******
**************************

ExportAGP.pl 11203 2020-06-18 21:28:10Z jwang

Running on host: 095a2fd4cb52

Process ID (PID): 997
Start time: 2023-01-26 19:57:01

COMMAND
perl /home/bionano/tools/access/1.0/../../pipeline/1.0/HybridScaffold/1.0/hybridScaffold.pl -c hybridScaffold_DLE1_config_082522.xml -b EXP_REFINEFINAL1.cmap -n 3217.hap2.hic.filtered.fasta -u CTTAAG -z cur_results_10236032_10236036.zip -w status_10236032_10236036.txt -M conflicts_cut_status_CTTAAG_10236032_10236036.txt -g -B 2 -N 2 -r /home/bionano/tools/access/1.0/../../pipeline/1.0/RefAligner/1.0/RefAligner -p /home/bionano/tools/access/1.0/../../pipeline/1.0/Pipeline/1.0 -o /home/bionano/access/local/jobs/10236036/output

NGS file: /home/bionano/access/local/jobs/10236036/output/fa2cmap/3217.hap2.hic.filtered.fasta 
BNG file: /home/bionano/access/local/jobs/10236036/EXP_REFINEFINAL1.cmap 
Configuration file: /home/bionano/access/local/jobs/10236036/hybridScaffold_DLE1_config_082522.xml 
Output folder: /home/bionano/access/local/jobs/10236036/output
User-defined conflict status file: /home/bionano/access/local/jobs/10236036/conflicts_cut_status_CTTAAG_10236032_10236036.txt
RefAligner binary: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner

Beginning cut_conflicts based on input a conflict file...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/cut_conflicts.pl -oriGMFile /home/bionano/access/local/jobs/10236036/output/align0/EXP_REFINEFINAL1_bppAdjust.cmap -oriSeqFile /home/bionano/access/local/jobs/10236036/output/fa2cmap/3217.hap2.hic.filtered_CTTAAG_0kb_0labels.cmap -outDir /home/bionano/access/local/jobs/10236036/output/assignAlignType/cut_conflicts_M1 -outGMFile /home/bionano/access/local/jobs/10236036/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -outSeqFile /home/bionano/access/local/jobs/10236036/output/assignAlignType/cut_conflicts_M1/3217.hap2.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -modBkptStatusFile /home/bionano/access/local/jobs/10236036/conflicts_cut_status_CTTAAG_10236032_10236036.txt -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
cut_conflicts complete in 4 seconds.
23 BNG contigs are found after the cut-conflict step
91 NGS contigs are found after the cut-conflict step

Beginning MergeNGS_BN...
*Using user-defined conflict-cut BioNano and sequence CMAP files*
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/MergeNGS_BN.pl -outputDir /home/bionano/access/local/jobs/10236036/output/mergeNGS_BN_M1 -refaligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -logFile mergeNGS_BN_script.log -ngs_cmap_fn /home/bionano/access/local/jobs/10236036/output/assignAlignType/cut_conflicts_M1/3217.hap2.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -bng_cmap_fn /home/bionano/access/local/jobs/10236036/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -id_shift 100000 -readparameters /home/bionano/access/local/jobs/10236036/output/align1/align1.errbin -xmlFile /home/bionano/access/local/jobs/10236036/hybridScaffold_DLE1_config_082522.xml
MergeNGS_BN complete in 25 seconds.

Beginning extraction of used and not used NGS contigs in hybrid scaffold pairmerge step...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/find_used_not_used_ngs.pl /home/bionano/access/local/jobs/10236036/output/mergeNGS_BN_M1 100000 /home/bionano/access/local/jobs/10236036/output/assignAlignType/cut_conflicts_M1/3217.hap2.hic.filtered_CTTAAG_0kb_0labels_cut.cmap /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
Extraction of NGS contigs complete in 1 second.

Beginning alignment of NGS cmap to Hybrid CMAP...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/align_final_seq_two_passes.pl -outDir /home/bionano/access/local/jobs/10236036/output/align_final_M1 -outFilePrefix EXP_REFINEFINAL1_bppAdjust_cmap_3217_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -xmlFile /home/bionano/access/local/jobs/10236036/hybridScaffold_DLE1_config_082522.xml -seqFile /home/bionano/access/local/jobs/10236036/output/assignAlignType/cut_conflicts_M1/3217.hap2.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -scaffoldFile /home/bionano/access/local/jobs/10236036/output/align_final_M1/step2.hybrid.cmap -logFile /home/bionano/access/local/jobs/10236036/output/align_final_M1/align_final_seq_script_log.txt
align sequences to hybrid scaffolds completed  in 47 seconds.

Beginning extraction of used and not used BNG contigs in hybrid scaffold pairmerge step...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/find_used_not_used_bn.pl /home/bionano/access/local/jobs/10236036/output/mergeNGS_BN_M1 100000 /home/bionano/access/local/jobs/10236036/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
Extraction of BNG contigs complete in 1 second.

Beginning alignment of BNG cmap to Hybrid CMAP...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/align_final_bng.pl -outDir /home/bionano/access/local/jobs/10236036/output/align_final_M1 -outFilePrefix EXP_REFINEFINAL1_bppAdjust_cmap_3217_hap2_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -xmlFile /home/bionano/access/local/jobs/10236036/hybridScaffold_DLE1_config_082522.xml -bngFile /home/bionano/access/local/jobs/10236036/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -scaffoldFile /home/bionano/access/local/jobs/10236036/output/align_final_M1/step2.hybrid.cmap -logFile /home/bionano/access/local/jobs/10236036/output/align_final_M1/align_final_bng_log.txt
align Bionano genome maps to hybrid scaffolds completed  in 1 minute and 6 seconds.

Merging Hybrid CMAP with NGS not participated in the hybrid scaffold...
Running command: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -f -i /home/bionano/access/local/jobs/10236036/output/align_final_M1/step2.hybrid.cmap -i /home/bionano/access/local/jobs/10236036/output/mergeNGS_BN_M1/all_not_used_ngs.cmap -o HYBRID_SCAFFOLD_notUsedNGS_merged -merge -minsites 0 -stdout -stderr
Merging Hybrid CMAP with naive NGS CMAP complete in no time.

Merging Hybrid CMAP with BNG CMAP not participated in the hybrid scaffold...
Running command: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -f -i /home/bionano/access/local/jobs/10236036/output/align_final_M1/step2.hybrid.cmap -i /home/bionano/access/local/jobs/10236036/output/mergeNGS_BN_M1/all_not_used_bng.cmap -o HYBRID_SCAFFOLD_notUsedBNG_merged -merge -stdout -stderr
Merging Hybrid CMAP with naive BNG CMAP complete in 1 second.

Beginnning construction of AGP and FASTA file of the scaffolded and unscaffolded sequences...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/ExportAGP.pl -i /home/bionano/access/local/jobs/10236036/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_3217_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap -c /home/bionano/access/local/jobs/10236036/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_3217_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_r.cmap -o /home/bionano/access/local/jobs/10236036/output/agp_fasta_M1 -m /home/bionano/access/local/jobs/10236036/output/fa2cmap/3217.hap2.hic.filtered_CTTAAG_0kb_0labels_key.txt -s /home/bionano/access/local/jobs/10236036/output/fa2cmap/3217.hap2.hic.filtered.fasta -e CTTAAG 1 -r /home/bionano/access/local/jobs/10236036/output/assignAlignType/cut_conflicts_M1/auto_cut_NGS_coord_translation.txt -t
AGP and FASTA generation complete in 43 seconds.

Calculating statistics...

Original BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236036/EXP_REFINEFINAL1.cmap

Count  = 24
Min length (Mbp) = 0.119
Median length (Mbp) = 15.781
Mean length (Mbp) = 20.226
N50 length (Mbp) = 35.578
Max length (Mbp) = 49.423
Total length (Mbp) = 485.412

Bpp-adjusted BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236036/output/align0/EXP_REFINEFINAL1_bppAdjust.cmap

Count  = 24
Min length (Mbp) = 0.119
Median length (Mbp) = 15.772
Mean length (Mbp) = 20.214
N50 length (Mbp) = 35.558
Max length (Mbp) = 49.394
Total length (Mbp) = 485.133

Original NGS sequences statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236036/output/fa2cmap/3217.hap2.hic.filtered_CTTAAG_0kb_0labels.cmap

Count  = 109
Min length (Mbp) = 0.070
Median length (Mbp) = 0.276
Mean length (Mbp) = 4.599
N50 length (Mbp) = 23.539
Max length (Mbp) = 45.444
Total length (Mbp) = 501.264

Conflict Resolution from BNG-NGS alignment:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_conflicts_cut_stats.pl /home/bionano/access/local/jobs/10236036/output/assignAlignType/cut_conflicts_M1/conflicts_cut_status.txt

Number of conflict cuts made to Bionano maps: 0
Number of conflict cuts made to NGS sequences: 0
Number of Bionano maps to be cut: 0
Number of NGS sequences to be cut: 0

Before merge: BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236036/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap

Count  = 23
Min length (Mbp) = 0.119
Median length (Mbp) = 17.060
Mean length (Mbp) = 21.058
N50 length (Mbp) = 35.558
Max length (Mbp) = 49.394
Total length (Mbp) = 484.326

Before merge: NGS sequences statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236036/output/assignAlignType/cut_conflicts_M1/3217.hap2.hic.filtered_CTTAAG_0kb_0labels_cut.cmap

Count  = 91
Min length (Mbp) = 0.070
Median length (Mbp) = 0.244
Mean length (Mbp) = 5.444
N50 length (Mbp) = 23.539
Max length (Mbp) = 45.444
Total length (Mbp) = 495.371

BNG Genome Map in hybrid scaffold statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236036/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_3217_hap2_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD_q.cmap

Count  = 22
Min length (Mbp) = 0.119
Median length (Mbp) = 18.810
Mean length (Mbp) = 22.008
N50 length (Mbp) = 35.558
Max length (Mbp) = 49.394
Total length (Mbp) = 484.167

NGS sequences in hybrid scaffold (CMAP) statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236036/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_3217_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_q.cmap

Count  = 35
Min length (Mbp) = 0.176
Median length (Mbp) = 9.841
Mean length (Mbp) = 13.792
N50 length (Mbp) = 29.952
Max length (Mbp) = 45.444
Total length (Mbp) = 482.720

Hybrid scaffold Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236036/output/align_final_M1/step2.hybrid.cmap

Count  = 13
Min length (Mbp) = 5.171
Median length (Mbp) = 42.439
Mean length (Mbp) = 37.209
N50 length (Mbp) = 43.714
Max length (Mbp) = 51.448
Total length (Mbp) = 483.722

NGS FASTA sequence in hybrid scaffold statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_scaffolded_seq_fasta_stats.pl /home/bionano/access/local/jobs/10236036/output/agp_fasta_M1/3217.hap2.hic.filtered.fasta.cut.fasta /home/bionano/access/local/jobs/10236036/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_3217_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_NOT_SCAFFOLDED.fasta

Count  = 35
Min length (Mbp) = 0.176
Median length (Mbp) = 9.841
Mean length (Mbp) = 13.782
N50 length (Mbp) = 29.933
Max length (Mbp) = 45.444
Total length (Mbp) = 482.375

Hybrid scaffold FASTA statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_fasta_stats.pl /home/bionano/access/local/jobs/10236036/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_3217_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.fasta

Count  = 13
Min length (Mbp) = 4.926
Median length (Mbp) = 42.373
Mean length (Mbp) = 37.140
N50 length (Mbp) = 43.695
Max length (Mbp) = 51.439
Total length (Mbp) = 482.816

Hybrid scaffold FASTA plus not scaffolded NGS FASTA statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_hybrid_scaffold_not_scaffolded_seq_fasta_stats.pl /home/bionano/access/local/jobs/10236036/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_3217_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.fasta /home/bionano/access/local/jobs/10236036/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_3217_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_NOT_SCAFFOLDED.fasta

Count  = 102
Min length (Mbp) = 0.002
Median length (Mbp) = 0.181
Mean length (Mbp) = 4.919
N50 length (Mbp) = 43.695
Max length (Mbp) = 51.439
Total length (Mbp) = 501.706


Calculating CMAP statistics complete in 16 seconds.

Calculating XMAP statistics...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_xmap_stats.pl /home/bionano/access/local/jobs/10236036/output/align_final_M1 EXP_REFINEFINAL1_bppAdjust_cmap_3217_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap step2.hybrid.cmap EXP_REFINEFINAL1_bppAdjust_cmap_3217_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap.stats

refContig	refContig_size	query_cov_bp	query_unique_cov_bp	query_unique_cov_percent
4	34908533.8	34862107.4	34862107.4	99.87
9	42438692.2	42377609	42364457.5	99.83
100017	5171326.9	5159727.1	5004905.1	96.78
5	44151720.5	44103922.7	44103922.7	99.89
15	12091966.0	12068812.5	12068812.5	99.81
2	35564285.5	35438811.7	35438811.7	99.65
100001	49394401.6	49227254.1	49207821.1	99.62
3	35834356.0	35691423.3	35680952	99.57
10	49234491.6	49181409.8	49162124.6	99.85
100009	34325577.8	34221030.6	34212160	99.67
100003	43714239.6	43219888.8	43195093.6	98.81
1	45444082.0	45373842	45373842	99.85
6	51447959.4	51327125.4	51327125.4	99.77

Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_xmap_stats.pl /home/bionano/access/local/jobs/10236036/output/align_final_M1 EXP_REFINEFINAL1_bppAdjust_cmap_3217_hap2_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD.xmap step2.hybrid.cmap EXP_REFINEFINAL1_bppAdjust_cmap_3217_hap2_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD.xmap.stats

refContig	refContig_size	query_cov_bp	query_unique_cov_bp	query_unique_cov_percent
100009	34325577.8	34263072.9	34263072.9	99.82
1	45444082.0	45355560	45355560	99.81
100001	49394401.6	49346843.7	49346843.7	99.90
4	34908533.8	34862107.4	34862107.4	99.87
3	35834356.0	35764781.8	35764781.8	99.81
15	12091966.0	12068812.5	12068812.5	99.81
10	49234491.6	49167788.3	49167788.3	99.86
5	44151720.5	43766805.9	43693163.9	98.96
9	42438692.2	42368145.6	42368145.6	99.83
100017	5171326.9	5108646.9	5108646.9	98.79
6	51447959.4	51477575.2	51338431.2	99.79
100003	43714239.6	43635214.1	43635214.1	99.82
2	35564285.5	35438811.7	35438811.7	99.65

XMAP statistics calculation complete in no time.

Beginning archive and zip of result files...
