**************************
*****BioNano Genomics*****
******BNG-NGS Merge*******
**************************

ExportAGP.pl 11203 2020-06-18 21:28:10Z jwang

Running on host: 071658a36f85

Process ID (PID): 997
Start time: 2023-01-31 19:07:58

COMMAND
perl /home/bionano/tools/access/1.0/../../pipeline/1.0/HybridScaffold/1.0/hybridScaffold.pl -c hybridScaffold_DLE1_config_082522.xml -b EXP_REFINEFINAL1.cmap -n SH3142.hap2.hic.filtered.fasta -u CTTAAG -z cur_results_10236019_10236354.zip -w status_10236019_10236354.txt -M conflicts_cut_status_CTTAAG_10236019_10236354.txt -g -B 2 -N 2 -r /home/bionano/tools/access/1.0/../../pipeline/1.0/RefAligner/1.0/RefAligner -p /home/bionano/tools/access/1.0/../../pipeline/1.0/Pipeline/1.0 -o /home/bionano/access/local/jobs/10236354/output

NGS file: /home/bionano/access/local/jobs/10236354/output/fa2cmap/SH3142.hap2.hic.filtered.fasta 
BNG file: /home/bionano/access/local/jobs/10236354/EXP_REFINEFINAL1.cmap 
Configuration file: /home/bionano/access/local/jobs/10236354/hybridScaffold_DLE1_config_082522.xml 
Output folder: /home/bionano/access/local/jobs/10236354/output
User-defined conflict status file: /home/bionano/access/local/jobs/10236354/conflicts_cut_status_CTTAAG_10236019_10236354.txt
RefAligner binary: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner

Beginning cut_conflicts based on input a conflict file...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/cut_conflicts.pl -oriGMFile /home/bionano/access/local/jobs/10236354/output/align0/EXP_REFINEFINAL1_bppAdjust.cmap -oriSeqFile /home/bionano/access/local/jobs/10236354/output/fa2cmap/SH3142.hap2.hic.filtered_CTTAAG_0kb_0labels.cmap -outDir /home/bionano/access/local/jobs/10236354/output/assignAlignType/cut_conflicts_M1 -outGMFile /home/bionano/access/local/jobs/10236354/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -outSeqFile /home/bionano/access/local/jobs/10236354/output/assignAlignType/cut_conflicts_M1/SH3142.hap2.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -modBkptStatusFile /home/bionano/access/local/jobs/10236354/conflicts_cut_status_CTTAAG_10236019_10236354.txt -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
cut_conflicts complete in 4 seconds.
19 BNG contigs are found after the cut-conflict step
125 NGS contigs are found after the cut-conflict step

Beginning MergeNGS_BN...
*Using user-defined conflict-cut BioNano and sequence CMAP files*
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/MergeNGS_BN.pl -outputDir /home/bionano/access/local/jobs/10236354/output/mergeNGS_BN_M1 -refaligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -logFile mergeNGS_BN_script.log -ngs_cmap_fn /home/bionano/access/local/jobs/10236354/output/assignAlignType/cut_conflicts_M1/SH3142.hap2.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -bng_cmap_fn /home/bionano/access/local/jobs/10236354/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -id_shift 100000 -readparameters /home/bionano/access/local/jobs/10236354/output/align1/align1.errbin -xmlFile /home/bionano/access/local/jobs/10236354/hybridScaffold_DLE1_config_082522.xml
MergeNGS_BN complete in 27 seconds.

Beginning extraction of used and not used NGS contigs in hybrid scaffold pairmerge step...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/find_used_not_used_ngs.pl /home/bionano/access/local/jobs/10236354/output/mergeNGS_BN_M1 100000 /home/bionano/access/local/jobs/10236354/output/assignAlignType/cut_conflicts_M1/SH3142.hap2.hic.filtered_CTTAAG_0kb_0labels_cut.cmap /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
Extraction of NGS contigs complete in 1 second.

Beginning alignment of NGS cmap to Hybrid CMAP...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/align_final_seq_two_passes.pl -outDir /home/bionano/access/local/jobs/10236354/output/align_final_M1 -outFilePrefix EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -xmlFile /home/bionano/access/local/jobs/10236354/hybridScaffold_DLE1_config_082522.xml -seqFile /home/bionano/access/local/jobs/10236354/output/assignAlignType/cut_conflicts_M1/SH3142.hap2.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -scaffoldFile /home/bionano/access/local/jobs/10236354/output/align_final_M1/step2.hybrid.cmap -logFile /home/bionano/access/local/jobs/10236354/output/align_final_M1/align_final_seq_script_log.txt
align sequences to hybrid scaffolds completed  in 44 seconds.

Beginning extraction of used and not used BNG contigs in hybrid scaffold pairmerge step...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/find_used_not_used_bn.pl /home/bionano/access/local/jobs/10236354/output/mergeNGS_BN_M1 100000 /home/bionano/access/local/jobs/10236354/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
Extraction of BNG contigs complete in 2 seconds.

Beginning alignment of BNG cmap to Hybrid CMAP...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/align_final_bng.pl -outDir /home/bionano/access/local/jobs/10236354/output/align_final_M1 -outFilePrefix EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap2_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -xmlFile /home/bionano/access/local/jobs/10236354/hybridScaffold_DLE1_config_082522.xml -bngFile /home/bionano/access/local/jobs/10236354/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -scaffoldFile /home/bionano/access/local/jobs/10236354/output/align_final_M1/step2.hybrid.cmap -logFile /home/bionano/access/local/jobs/10236354/output/align_final_M1/align_final_bng_log.txt
align Bionano genome maps to hybrid scaffolds completed  in 32 seconds.

Merging Hybrid CMAP with NGS not participated in the hybrid scaffold...
Running command: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -f -i /home/bionano/access/local/jobs/10236354/output/align_final_M1/step2.hybrid.cmap -i /home/bionano/access/local/jobs/10236354/output/mergeNGS_BN_M1/all_not_used_ngs.cmap -o HYBRID_SCAFFOLD_notUsedNGS_merged -merge -minsites 0 -stdout -stderr
Merging Hybrid CMAP with naive NGS CMAP complete in 1 second.

Merging Hybrid CMAP with BNG CMAP not participated in the hybrid scaffold...
Running command: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -f -i /home/bionano/access/local/jobs/10236354/output/align_final_M1/step2.hybrid.cmap -i /home/bionano/access/local/jobs/10236354/output/mergeNGS_BN_M1/all_not_used_bng.cmap -o HYBRID_SCAFFOLD_notUsedBNG_merged -merge -stdout -stderr
Merging Hybrid CMAP with naive BNG CMAP complete in 1 second.

Beginnning construction of AGP and FASTA file of the scaffolded and unscaffolded sequences...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/ExportAGP.pl -i /home/bionano/access/local/jobs/10236354/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap -c /home/bionano/access/local/jobs/10236354/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_r.cmap -o /home/bionano/access/local/jobs/10236354/output/agp_fasta_M1 -m /home/bionano/access/local/jobs/10236354/output/fa2cmap/SH3142.hap2.hic.filtered_CTTAAG_0kb_0labels_key.txt -s /home/bionano/access/local/jobs/10236354/output/fa2cmap/SH3142.hap2.hic.filtered.fasta -e CTTAAG 1 -r /home/bionano/access/local/jobs/10236354/output/assignAlignType/cut_conflicts_M1/auto_cut_NGS_coord_translation.txt -t
AGP and FASTA generation complete in 46 seconds.

Calculating statistics...

Original BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236354/EXP_REFINEFINAL1.cmap

Count  = 19
Min length (Mbp) = 0.768
Median length (Mbp) = 30.000
Mean length (Mbp) = 26.106
N50 length (Mbp) = 39.943
Max length (Mbp) = 49.005
Total length (Mbp) = 496.006

Bpp-adjusted BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236354/output/align0/EXP_REFINEFINAL1_bppAdjust.cmap

Count  = 19
Min length (Mbp) = 0.768
Median length (Mbp) = 29.984
Mean length (Mbp) = 26.092
N50 length (Mbp) = 39.922
Max length (Mbp) = 48.979
Total length (Mbp) = 495.749

Original NGS sequences statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236354/output/fa2cmap/SH3142.hap2.hic.filtered_CTTAAG_0kb_0labels.cmap

Count  = 127
Min length (Mbp) = 0.071
Median length (Mbp) = 0.269
Mean length (Mbp) = 4.025
N50 length (Mbp) = 18.196
Max length (Mbp) = 42.956
Total length (Mbp) = 511.147

Conflict Resolution from BNG-NGS alignment:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_conflicts_cut_stats.pl /home/bionano/access/local/jobs/10236354/output/assignAlignType/cut_conflicts_M1/conflicts_cut_status.txt

Number of conflict cuts made to Bionano maps: 0
Number of conflict cuts made to NGS sequences: 1
Number of Bionano maps to be cut: 0
Number of NGS sequences to be cut: 1

Before merge: BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236354/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap

Count  = 19
Min length (Mbp) = 0.768
Median length (Mbp) = 29.984
Mean length (Mbp) = 26.092
N50 length (Mbp) = 39.922
Max length (Mbp) = 48.979
Total length (Mbp) = 495.749

Before merge: NGS sequences statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236354/output/assignAlignType/cut_conflicts_M1/SH3142.hap2.hic.filtered_CTTAAG_0kb_0labels_cut.cmap

Count  = 125
Min length (Mbp) = 0.071
Median length (Mbp) = 0.270
Mean length (Mbp) = 4.083
N50 length (Mbp) = 18.196
Max length (Mbp) = 42.956
Total length (Mbp) = 510.326

BNG Genome Map in hybrid scaffold statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236354/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap2_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD_q.cmap

Count  = 17
Min length (Mbp) = 3.843
Median length (Mbp) = 34.367
Mean length (Mbp) = 29.064
N50 length (Mbp) = 39.922
Max length (Mbp) = 48.979
Total length (Mbp) = 494.086

NGS sequences in hybrid scaffold (CMAP) statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236354/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_q.cmap

Count  = 50
Min length (Mbp) = 0.084
Median length (Mbp) = 7.364
Mean length (Mbp) = 9.913
N50 length (Mbp) = 18.196
Max length (Mbp) = 42.956
Total length (Mbp) = 495.673

Hybrid scaffold Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236354/output/align_final_M1/step2.hybrid.cmap

Count  = 16
Min length (Mbp) = 0.412
Median length (Mbp) = 36.538
Mean length (Mbp) = 31.194
N50 length (Mbp) = 42.965
Max length (Mbp) = 49.874
Total length (Mbp) = 499.109

NGS FASTA sequence in hybrid scaffold statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_scaffolded_seq_fasta_stats.pl /home/bionano/access/local/jobs/10236354/output/agp_fasta_M1/SH3142.hap2.hic.filtered.fasta.cut.fasta /home/bionano/access/local/jobs/10236354/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_NOT_SCAFFOLDED.fasta

Count  = 48
Min length (Mbp) = 0.084
Median length (Mbp) = 8.352
Mean length (Mbp) = 10.317
N50 length (Mbp) = 18.196
Max length (Mbp) = 42.956
Total length (Mbp) = 495.232

Hybrid scaffold FASTA statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_fasta_stats.pl /home/bionano/access/local/jobs/10236354/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.fasta

Count  = 16
Min length (Mbp) = 0.412
Median length (Mbp) = 36.172
Mean length (Mbp) = 30.983
N50 length (Mbp) = 42.956
Max length (Mbp) = 49.846
Total length (Mbp) = 495.732

Hybrid scaffold FASTA plus not scaffolded NGS FASTA statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_hybrid_scaffold_not_scaffolded_seq_fasta_stats.pl /home/bionano/access/local/jobs/10236354/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.fasta /home/bionano/access/local/jobs/10236354/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_NOT_SCAFFOLDED.fasta

Count  = 109
Min length (Mbp) = 0.000
Median length (Mbp) = 0.127
Mean length (Mbp) = 4.694
N50 length (Mbp) = 42.956
Max length (Mbp) = 49.846
Total length (Mbp) = 511.647


Calculating CMAP statistics complete in 16 seconds.

Calculating XMAP statistics...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_xmap_stats.pl /home/bionano/access/local/jobs/10236354/output/align_final_M1 EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap step2.hybrid.cmap EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap.stats

refContig	refContig_size	query_cov_bp	query_unique_cov_bp	query_unique_cov_percent
100015	8772224.2	8553841.4	8553841.4	97.51
100016	6693053.3	6427326.5	6427326.5	96.03
100012	17779623.0	16571509.4	16571509.4	93.21
100001	48979400.0	48826288.3	48818131.2	99.67
7	24291740.8	23643534.1	23630418.1	97.28
51	411576.0	586056	406650	98.80
100009	34367321.9	34243037.3	34243037.3	99.64
16	13033127.8	12763255.9	12763255.9	97.93
1	42965243.8	42825564.7	42825564.7	99.67
4	49873999.8	49776533.1	49776533.1	99.80
13	47501906.0	47432646.4	47432646.4	99.85
100010	29983976.4	29790776.1	29790776.1	99.36
5	48358018.1	48299336.6	48275203.3	99.83
31	39925998.1	39799446.7	39799446.7	99.68
20	38709646.2	37610338.1	37601449	97.14
29	47461834.9	47341039.4	47318753.6	99.70

Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_xmap_stats.pl /home/bionano/access/local/jobs/10236354/output/align_final_M1 EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap2_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD.xmap step2.hybrid.cmap EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap2_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD.xmap.stats

refContig	refContig_size	query_cov_bp	query_unique_cov_bp	query_unique_cov_percent
13	47501906.0	47449280.1	47449280.1	99.89
100009	34367321.9	34266940	34266940	99.71
29	47461834.9	43031495.2	43031495.2	90.67
16	13033127.8	12871192.6	12871192.6	98.76
51	411576.0	0	0	0.00
7	24291740.8	24568689.5	24263416.5	99.88
5	48358018.1	48293833.1	48293833.1	99.87
31	39925998.1	39880805.6	39880805.6	99.89
100015	8772224.2	8705021.6	8705021.6	99.23
4	49873999.8	49369897.3	49369897.3	98.99
100016	6693053.3	6657896	6657896	99.47
1	42965243.8	42825564.7	42825564.7	99.67
100010	29983976.4	29939493.7	29939493.7	99.85
100001	48979400.0	48930331.9	48930331.9	99.90
100012	17779623.0	17738460	17738460	99.77
20	38709646.2	38658766.6	38658766.6	99.87

XMAP statistics calculation complete in no time.

Beginning archive and zip of result files...
