**************************
*****BioNano Genomics*****
******BNG-NGS Merge*******
**************************

ExportAGP.pl 11203 2020-06-18 21:28:10Z jwang

Running on host: 6d5379c5276e

Process ID (PID): 997
Start time: 2023-01-31 19:05:03

COMMAND
perl /home/bionano/tools/access/1.0/../../pipeline/1.0/HybridScaffold/1.0/hybridScaffold.pl -c hybridScaffold_DLE1_config_082522.xml -b EXP_REFINEFINAL1.cmap -n SH3142.hap1.hic.filtered.fasta -u CTTAAG -z cur_results_10236018_10236353.zip -w status_10236018_10236353.txt -M conflicts_cut_status_CTTAAG_10236018_10236353.txt -g -B 2 -N 2 -r /home/bionano/tools/access/1.0/../../pipeline/1.0/RefAligner/1.0/RefAligner -p /home/bionano/tools/access/1.0/../../pipeline/1.0/Pipeline/1.0 -o /home/bionano/access/local/jobs/10236353/output

NGS file: /home/bionano/access/local/jobs/10236353/output/fa2cmap/SH3142.hap1.hic.filtered.fasta 
BNG file: /home/bionano/access/local/jobs/10236353/EXP_REFINEFINAL1.cmap 
Configuration file: /home/bionano/access/local/jobs/10236353/hybridScaffold_DLE1_config_082522.xml 
Output folder: /home/bionano/access/local/jobs/10236353/output
User-defined conflict status file: /home/bionano/access/local/jobs/10236353/conflicts_cut_status_CTTAAG_10236018_10236353.txt
RefAligner binary: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner

Beginning cut_conflicts based on input a conflict file...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/cut_conflicts.pl -oriGMFile /home/bionano/access/local/jobs/10236353/output/align0/EXP_REFINEFINAL1_bppAdjust.cmap -oriSeqFile /home/bionano/access/local/jobs/10236353/output/fa2cmap/SH3142.hap1.hic.filtered_CTTAAG_0kb_0labels.cmap -outDir /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1 -outGMFile /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -outSeqFile /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/SH3142.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -modBkptStatusFile /home/bionano/access/local/jobs/10236353/conflicts_cut_status_CTTAAG_10236018_10236353.txt -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
cut_conflicts complete in 3 seconds.
21 BNG contigs are found after the cut-conflict step
124 NGS contigs are found after the cut-conflict step

Beginning MergeNGS_BN...
*Using user-defined conflict-cut BioNano and sequence CMAP files*
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/MergeNGS_BN.pl -outputDir /home/bionano/access/local/jobs/10236353/output/mergeNGS_BN_M1 -refaligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -logFile mergeNGS_BN_script.log -ngs_cmap_fn /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/SH3142.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -bng_cmap_fn /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -id_shift 100000 -readparameters /home/bionano/access/local/jobs/10236353/output/align1/align1.errbin -xmlFile /home/bionano/access/local/jobs/10236353/hybridScaffold_DLE1_config_082522.xml
MergeNGS_BN complete in 22 seconds.

Beginning extraction of used and not used NGS contigs in hybrid scaffold pairmerge step...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/find_used_not_used_ngs.pl /home/bionano/access/local/jobs/10236353/output/mergeNGS_BN_M1 100000 /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/SH3142.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
Extraction of NGS contigs complete in 1 second.

Beginning alignment of NGS cmap to Hybrid CMAP...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/align_final_seq_two_passes.pl -outDir /home/bionano/access/local/jobs/10236353/output/align_final_M1 -outFilePrefix EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -xmlFile /home/bionano/access/local/jobs/10236353/hybridScaffold_DLE1_config_082522.xml -seqFile /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/SH3142.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -scaffoldFile /home/bionano/access/local/jobs/10236353/output/align_final_M1/step2.hybrid.cmap -logFile /home/bionano/access/local/jobs/10236353/output/align_final_M1/align_final_seq_script_log.txt
align sequences to hybrid scaffolds completed  in 51 seconds.

Beginning extraction of used and not used BNG contigs in hybrid scaffold pairmerge step...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/find_used_not_used_bn.pl /home/bionano/access/local/jobs/10236353/output/mergeNGS_BN_M1 100000 /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
Extraction of BNG contigs complete in 1 second.

Beginning alignment of BNG cmap to Hybrid CMAP...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/align_final_bng.pl -outDir /home/bionano/access/local/jobs/10236353/output/align_final_M1 -outFilePrefix EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -xmlFile /home/bionano/access/local/jobs/10236353/hybridScaffold_DLE1_config_082522.xml -bngFile /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -scaffoldFile /home/bionano/access/local/jobs/10236353/output/align_final_M1/step2.hybrid.cmap -logFile /home/bionano/access/local/jobs/10236353/output/align_final_M1/align_final_bng_log.txt
align Bionano genome maps to hybrid scaffolds completed  in 34 seconds.

Merging Hybrid CMAP with NGS not participated in the hybrid scaffold...
Running command: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -f -i /home/bionano/access/local/jobs/10236353/output/align_final_M1/step2.hybrid.cmap -i /home/bionano/access/local/jobs/10236353/output/mergeNGS_BN_M1/all_not_used_ngs.cmap -o HYBRID_SCAFFOLD_notUsedNGS_merged -merge -minsites 0 -stdout -stderr
Merging Hybrid CMAP with naive NGS CMAP complete in 1 second.

Merging Hybrid CMAP with BNG CMAP not participated in the hybrid scaffold...
Running command: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -f -i /home/bionano/access/local/jobs/10236353/output/align_final_M1/step2.hybrid.cmap -i /home/bionano/access/local/jobs/10236353/output/mergeNGS_BN_M1/all_not_used_bng.cmap -o HYBRID_SCAFFOLD_notUsedBNG_merged -merge -stdout -stderr
Merging Hybrid CMAP with naive BNG CMAP complete in 1 second.

Beginnning construction of AGP and FASTA file of the scaffolded and unscaffolded sequences...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/ExportAGP.pl -i /home/bionano/access/local/jobs/10236353/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap -c /home/bionano/access/local/jobs/10236353/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_r.cmap -o /home/bionano/access/local/jobs/10236353/output/agp_fasta_M1 -m /home/bionano/access/local/jobs/10236353/output/fa2cmap/SH3142.hap1.hic.filtered_CTTAAG_0kb_0labels_key.txt -s /home/bionano/access/local/jobs/10236353/output/fa2cmap/SH3142.hap1.hic.filtered.fasta -e CTTAAG 1 -r /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/auto_cut_NGS_coord_translation.txt -t
AGP and FASTA generation complete in 44 seconds.

Calculating statistics...

Original BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236353/EXP_REFINEFINAL1.cmap

Count  = 21
Min length (Mbp) = 0.121
Median length (Mbp) = 20.793
Mean length (Mbp) = 22.985
N50 length (Mbp) = 45.663
Max length (Mbp) = 50.281
Total length (Mbp) = 482.679

Bpp-adjusted BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236353/output/align0/EXP_REFINEFINAL1_bppAdjust.cmap

Count  = 21
Min length (Mbp) = 0.121
Median length (Mbp) = 20.783
Mean length (Mbp) = 22.974
N50 length (Mbp) = 45.641
Max length (Mbp) = 50.257
Total length (Mbp) = 482.448

Original NGS sequences statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236353/output/fa2cmap/SH3142.hap1.hic.filtered_CTTAAG_0kb_0labels.cmap

Count  = 128
Min length (Mbp) = 0.071
Median length (Mbp) = 0.192
Mean length (Mbp) = 3.868
N50 length (Mbp) = 24.395
Max length (Mbp) = 49.057
Total length (Mbp) = 495.040

Conflict Resolution from BNG-NGS alignment:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_conflicts_cut_stats.pl /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/conflicts_cut_status.txt

Number of conflict cuts made to Bionano maps: 0
Number of conflict cuts made to NGS sequences: 0
Number of Bionano maps to be cut: 0
Number of NGS sequences to be cut: 0

Before merge: BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap

Count  = 21
Min length (Mbp) = 0.121
Median length (Mbp) = 20.783
Mean length (Mbp) = 22.974
N50 length (Mbp) = 45.641
Max length (Mbp) = 50.257
Total length (Mbp) = 482.448

Before merge: NGS sequences statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/SH3142.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap

Count  = 124
Min length (Mbp) = 0.071
Median length (Mbp) = 0.187
Mean length (Mbp) = 3.982
N50 length (Mbp) = 24.395
Max length (Mbp) = 49.057
Total length (Mbp) = 493.730

BNG Genome Map in hybrid scaffold statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236353/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD_q.cmap

Count  = 15
Min length (Mbp) = 1.784
Median length (Mbp) = 36.761
Mean length (Mbp) = 31.756
N50 length (Mbp) = 46.663
Max length (Mbp) = 50.257
Total length (Mbp) = 476.335

NGS sequences in hybrid scaffold (CMAP) statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236353/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_q.cmap

Count  = 36
Min length (Mbp) = 0.094
Median length (Mbp) = 7.646
Mean length (Mbp) = 13.257
N50 length (Mbp) = 24.395
Max length (Mbp) = 49.057
Total length (Mbp) = 477.256

Hybrid scaffold Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236353/output/align_final_M1/step2.hybrid.cmap

Count  = 14
Min length (Mbp) = 1.941
Median length (Mbp) = 38.286
Mean length (Mbp) = 34.154
N50 length (Mbp) = 46.678
Max length (Mbp) = 50.257
Total length (Mbp) = 478.162

NGS FASTA sequence in hybrid scaffold statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_scaffolded_seq_fasta_stats.pl /home/bionano/access/local/jobs/10236353/output/agp_fasta_M1/SH3142.hap1.hic.filtered.fasta.cut.fasta /home/bionano/access/local/jobs/10236353/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_NOT_SCAFFOLDED.fasta

Count  = 36
Min length (Mbp) = 0.046
Median length (Mbp) = 7.635
Mean length (Mbp) = 13.252
N50 length (Mbp) = 24.391
Max length (Mbp) = 49.057
Total length (Mbp) = 477.060

Hybrid scaffold FASTA statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_fasta_stats.pl /home/bionano/access/local/jobs/10236353/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.fasta

Count  = 14
Min length (Mbp) = 1.914
Median length (Mbp) = 38.262
Mean length (Mbp) = 34.084
N50 length (Mbp) = 46.656
Max length (Mbp) = 50.211
Total length (Mbp) = 477.176

Hybrid scaffold FASTA plus not scaffolded NGS FASTA statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_hybrid_scaffold_not_scaffolded_seq_fasta_stats.pl /home/bionano/access/local/jobs/10236353/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.fasta /home/bionano/access/local/jobs/10236353/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_NOT_SCAFFOLDED.fasta

Count  = 119
Min length (Mbp) = 0.001
Median length (Mbp) = 0.133
Mean length (Mbp) = 4.161
N50 length (Mbp) = 45.603
Max length (Mbp) = 50.211
Total length (Mbp) = 495.156


Calculating CMAP statistics complete in 17 seconds.

Calculating XMAP statistics...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_xmap_stats.pl /home/bionano/access/local/jobs/10236353/output/align_final_M1 EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap step2.hybrid.cmap EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap.stats

refContig	refContig_size	query_cov_bp	query_unique_cov_bp	query_unique_cov_percent
3	39649759.7	39606637.4	39606637.4	99.89
14	46798738.2	46733828.4	46733828.4	99.86
100013	5301110.5	5059281.3	5059281.3	95.44
10	20790859.0	20700702.3	20700702.3	99.57
39	1941468.7	1920697.2	1835078.8	94.52
15	18085687.3	17846394.6	17846394.6	98.68
100009	35704529.1	35440046.7	35440046.7	99.26
4	47088199.1	47037292.4	47028708.4	99.87
12	46678111.9	46640736.8	46638028.4	99.91
11	45660567.5	45607206.4	45607206.4	99.88
100001	50256983.3	50114569.6	50107939.8	99.70
100002	49122649.6	49046225.2	49046225.2	99.84
8	36922738.7	36845612.3	36845612.3	99.79
100010	34160807.9	34093388	34085334.9	99.78

Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_xmap_stats.pl /home/bionano/access/local/jobs/10236353/output/align_final_M1 EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD.xmap step2.hybrid.cmap EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD.xmap.stats

refContig	refContig_size	query_cov_bp	query_unique_cov_bp	query_unique_cov_percent
100001	50256983.3	50188664.3	50188664.3	99.86
100002	49122649.6	49082889.7	49082889.7	99.92
39	1941468.7	1773716.8	1773716.8	91.36
15	18085687.3	16545972.3	16395671.3	90.66
12	46678111.9	46649368.8	46649368.8	99.94
10	20790859.0	20773370	20773370	99.92
11	45660567.5	45609950	45609950	99.89
100009	35704529.1	35658498.2	35658498.2	99.87
100013	5301110.5	5263815.4	5263815.4	99.30
100010	34160807.9	34104728.5	34104728.5	99.84
14	46798738.2	46761286	46761286	99.92
8	36922738.7	36897697.7	36897697.7	99.93
3	39649759.7	39606637.4	39606637.4	99.89
4	47088199.1	47046456.3	47046456.3	99.91

XMAP statistics calculation complete in 1 second.

Beginning archive and zip of result files...
