**************************
*****BioNano Genomics*****
******BNG-NGS Merge*******
**************************

ExportAGP.pl 11203 2020-06-18 21:28:10Z jwang

Running on host: c28294c3123f

Process ID (PID): 997
Start time: 2023-01-31 16:22:48

COMMAND
perl /home/bionano/tools/access/1.0/../../pipeline/1.0/HybridScaffold/1.0/hybridScaffold.pl -c hybridScaffold_DLE1_config_082522.xml -b EXP_REFINEFINAL1.cmap -n Maleb.hap2.hic.filtered.fasta -u CTTAAG -z cur_results_10236348_10236351.zip -w status_10236348_10236351.txt -M conflicts_cut_status_CTTAAG_10236348_10236351.txt -g -B 2 -N 2 -r /home/bionano/tools/access/1.0/../../pipeline/1.0/RefAligner/1.0/RefAligner -p /home/bionano/tools/access/1.0/../../pipeline/1.0/Pipeline/1.0 -o /home/bionano/access/local/jobs/10236351/output

NGS file: /home/bionano/access/local/jobs/10236351/output/fa2cmap/Maleb.hap2.hic.filtered.fasta 
BNG file: /home/bionano/access/local/jobs/10236351/EXP_REFINEFINAL1.cmap 
Configuration file: /home/bionano/access/local/jobs/10236351/hybridScaffold_DLE1_config_082522.xml 
Output folder: /home/bionano/access/local/jobs/10236351/output
User-defined conflict status file: /home/bionano/access/local/jobs/10236351/conflicts_cut_status_CTTAAG_10236348_10236351.txt
RefAligner binary: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner

Beginning cut_conflicts based on input a conflict file...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/cut_conflicts.pl -oriGMFile /home/bionano/access/local/jobs/10236351/output/align0/EXP_REFINEFINAL1_bppAdjust.cmap -oriSeqFile /home/bionano/access/local/jobs/10236351/output/fa2cmap/Maleb.hap2.hic.filtered_CTTAAG_0kb_0labels.cmap -outDir /home/bionano/access/local/jobs/10236351/output/assignAlignType/cut_conflicts_M1 -outGMFile /home/bionano/access/local/jobs/10236351/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -outSeqFile /home/bionano/access/local/jobs/10236351/output/assignAlignType/cut_conflicts_M1/Maleb.hap2.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -modBkptStatusFile /home/bionano/access/local/jobs/10236351/conflicts_cut_status_CTTAAG_10236348_10236351.txt -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
cut_conflicts complete in 3 seconds.
20 BNG contigs are found after the cut-conflict step
118 NGS contigs are found after the cut-conflict step

Beginning MergeNGS_BN...
*Using user-defined conflict-cut BioNano and sequence CMAP files*
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/MergeNGS_BN.pl -outputDir /home/bionano/access/local/jobs/10236351/output/mergeNGS_BN_M1 -refaligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -logFile mergeNGS_BN_script.log -ngs_cmap_fn /home/bionano/access/local/jobs/10236351/output/assignAlignType/cut_conflicts_M1/Maleb.hap2.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -bng_cmap_fn /home/bionano/access/local/jobs/10236351/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -id_shift 100000 -readparameters /home/bionano/access/local/jobs/10236351/output/align1/align1.errbin -xmlFile /home/bionano/access/local/jobs/10236351/hybridScaffold_DLE1_config_082522.xml
MergeNGS_BN complete in 21 seconds.

Beginning extraction of used and not used NGS contigs in hybrid scaffold pairmerge step...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/find_used_not_used_ngs.pl /home/bionano/access/local/jobs/10236351/output/mergeNGS_BN_M1 100000 /home/bionano/access/local/jobs/10236351/output/assignAlignType/cut_conflicts_M1/Maleb.hap2.hic.filtered_CTTAAG_0kb_0labels_cut.cmap /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
Extraction of NGS contigs complete in 1 second.

Beginning alignment of NGS cmap to Hybrid CMAP...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/align_final_seq_two_passes.pl -outDir /home/bionano/access/local/jobs/10236351/output/align_final_M1 -outFilePrefix EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -xmlFile /home/bionano/access/local/jobs/10236351/hybridScaffold_DLE1_config_082522.xml -seqFile /home/bionano/access/local/jobs/10236351/output/assignAlignType/cut_conflicts_M1/Maleb.hap2.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -scaffoldFile /home/bionano/access/local/jobs/10236351/output/align_final_M1/step2.hybrid.cmap -logFile /home/bionano/access/local/jobs/10236351/output/align_final_M1/align_final_seq_script_log.txt
align sequences to hybrid scaffolds completed  in 1 minute and 3 seconds.

Beginning extraction of used and not used BNG contigs in hybrid scaffold pairmerge step...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/find_used_not_used_bn.pl /home/bionano/access/local/jobs/10236351/output/mergeNGS_BN_M1 100000 /home/bionano/access/local/jobs/10236351/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
Extraction of BNG contigs complete in 1 second.

Beginning alignment of BNG cmap to Hybrid CMAP...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/align_final_bng.pl -outDir /home/bionano/access/local/jobs/10236351/output/align_final_M1 -outFilePrefix EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap2_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -xmlFile /home/bionano/access/local/jobs/10236351/hybridScaffold_DLE1_config_082522.xml -bngFile /home/bionano/access/local/jobs/10236351/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -scaffoldFile /home/bionano/access/local/jobs/10236351/output/align_final_M1/step2.hybrid.cmap -logFile /home/bionano/access/local/jobs/10236351/output/align_final_M1/align_final_bng_log.txt
align Bionano genome maps to hybrid scaffolds completed  in 39 seconds.

Merging Hybrid CMAP with NGS not participated in the hybrid scaffold...
Running command: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -f -i /home/bionano/access/local/jobs/10236351/output/align_final_M1/step2.hybrid.cmap -i /home/bionano/access/local/jobs/10236351/output/mergeNGS_BN_M1/all_not_used_ngs.cmap -o HYBRID_SCAFFOLD_notUsedNGS_merged -merge -minsites 0 -stdout -stderr
Merging Hybrid CMAP with naive NGS CMAP complete in 1 second.

Merging Hybrid CMAP with BNG CMAP not participated in the hybrid scaffold...
Running command: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -f -i /home/bionano/access/local/jobs/10236351/output/align_final_M1/step2.hybrid.cmap -i /home/bionano/access/local/jobs/10236351/output/mergeNGS_BN_M1/all_not_used_bng.cmap -o HYBRID_SCAFFOLD_notUsedBNG_merged -merge -stdout -stderr
Merging Hybrid CMAP with naive BNG CMAP complete in no time.

Beginnning construction of AGP and FASTA file of the scaffolded and unscaffolded sequences...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/ExportAGP.pl -i /home/bionano/access/local/jobs/10236351/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap -c /home/bionano/access/local/jobs/10236351/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_r.cmap -o /home/bionano/access/local/jobs/10236351/output/agp_fasta_M1 -m /home/bionano/access/local/jobs/10236351/output/fa2cmap/Maleb.hap2.hic.filtered_CTTAAG_0kb_0labels_key.txt -s /home/bionano/access/local/jobs/10236351/output/fa2cmap/Maleb.hap2.hic.filtered.fasta -e CTTAAG 1 -r /home/bionano/access/local/jobs/10236351/output/assignAlignType/cut_conflicts_M1/auto_cut_NGS_coord_translation.txt -t
AGP and FASTA generation complete in 44 seconds.

Calculating statistics...

Original BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236351/EXP_REFINEFINAL1.cmap

Count  = 20
Min length (Mbp) = 0.147
Median length (Mbp) = 24.377
Mean length (Mbp) = 23.516
N50 length (Mbp) = 35.363
Max length (Mbp) = 46.896
Total length (Mbp) = 470.329

Bpp-adjusted BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236351/output/align0/EXP_REFINEFINAL1_bppAdjust.cmap

Count  = 20
Min length (Mbp) = 0.147
Median length (Mbp) = 24.370
Mean length (Mbp) = 23.509
N50 length (Mbp) = 35.352
Max length (Mbp) = 46.881
Total length (Mbp) = 470.184

Original NGS sequences statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236351/output/fa2cmap/Maleb.hap2.hic.filtered_CTTAAG_0kb_0labels.cmap

Count  = 122
Min length (Mbp) = 0.074
Median length (Mbp) = 0.325
Mean length (Mbp) = 4.198
N50 length (Mbp) = 20.225
Max length (Mbp) = 38.683
Total length (Mbp) = 512.120

Conflict Resolution from BNG-NGS alignment:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_conflicts_cut_stats.pl /home/bionano/access/local/jobs/10236351/output/assignAlignType/cut_conflicts_M1/conflicts_cut_status.txt

Number of conflict cuts made to Bionano maps: 0
Number of conflict cuts made to NGS sequences: 2
Number of Bionano maps to be cut: 0
Number of NGS sequences to be cut: 1

Before merge: BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236351/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap

Count  = 20
Min length (Mbp) = 0.147
Median length (Mbp) = 24.370
Mean length (Mbp) = 23.509
N50 length (Mbp) = 35.352
Max length (Mbp) = 46.881
Total length (Mbp) = 470.184

Before merge: NGS sequences statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236351/output/assignAlignType/cut_conflicts_M1/Maleb.hap2.hic.filtered_CTTAAG_0kb_0labels_cut.cmap

Count  = 118
Min length (Mbp) = 0.074
Median length (Mbp) = 0.342
Mean length (Mbp) = 4.329
N50 length (Mbp) = 17.048
Max length (Mbp) = 38.683
Total length (Mbp) = 510.826

BNG Genome Map in hybrid scaffold statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236351/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap2_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD_q.cmap

Count  = 20
Min length (Mbp) = 0.147
Median length (Mbp) = 24.370
Mean length (Mbp) = 23.509
N50 length (Mbp) = 35.352
Max length (Mbp) = 46.881
Total length (Mbp) = 470.184

NGS sequences in hybrid scaffold (CMAP) statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236351/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_q.cmap

Count  = 59
Min length (Mbp) = 0.103
Median length (Mbp) = 4.533
Mean length (Mbp) = 8.064
N50 length (Mbp) = 17.048
Max length (Mbp) = 38.683
Total length (Mbp) = 475.780

Hybrid scaffold Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236351/output/align_final_M1/step2.hybrid.cmap

Count  = 24
Min length (Mbp) = 0.111
Median length (Mbp) = 15.249
Mean length (Mbp) = 19.845
N50 length (Mbp) = 38.527
Max length (Mbp) = 49.423
Total length (Mbp) = 476.275

NGS FASTA sequence in hybrid scaffold statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_scaffolded_seq_fasta_stats.pl /home/bionano/access/local/jobs/10236351/output/agp_fasta_M1/Maleb.hap2.hic.filtered.fasta.cut.fasta /home/bionano/access/local/jobs/10236351/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_NOT_SCAFFOLDED.fasta

Count  = 52
Min length (Mbp) = 0.003
Median length (Mbp) = 4.884
Mean length (Mbp) = 9.096
N50 length (Mbp) = 17.048
Max length (Mbp) = 38.683
Total length (Mbp) = 472.999

Hybrid scaffold FASTA statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_fasta_stats.pl /home/bionano/access/local/jobs/10236351/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.fasta

Count  = 24
Min length (Mbp) = 0.111
Median length (Mbp) = 15.232
Mean length (Mbp) = 19.714
N50 length (Mbp) = 38.087
Max length (Mbp) = 49.380
Total length (Mbp) = 473.130

Hybrid scaffold FASTA plus not scaffolded NGS FASTA statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_hybrid_scaffold_not_scaffolded_seq_fasta_stats.pl /home/bionano/access/local/jobs/10236351/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.fasta /home/bionano/access/local/jobs/10236351/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_NOT_SCAFFOLDED.fasta

Count  = 109
Min length (Mbp) = 0.005
Median length (Mbp) = 0.223
Mean length (Mbp) = 4.700
N50 length (Mbp) = 38.087
Max length (Mbp) = 49.380
Total length (Mbp) = 512.252


Calculating CMAP statistics complete in 16 seconds.

Calculating XMAP statistics...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_xmap_stats.pl /home/bionano/access/local/jobs/10236351/output/align_final_M1 EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap step2.hybrid.cmap EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap.stats

refContig	refContig_size	query_cov_bp	query_unique_cov_bp	query_unique_cov_percent
28	38526916.3	38353408.7	38353408.7	99.55
36	1337101.0	1599748	1326044	99.17
100009	31504095.7	31068182.6	31068182.6	98.62
3	48668124.1	48628599.1	48628599.1	99.92
108	110801.0	167317	107768	97.26
100007	33971428.0	33887364.6	33864831.6	99.69
15	49422970.2	49333535.7	49333535.7	99.82
39	1178039.0	1817359	1172335	99.52
52	474924.0	456017	456017	96.02
13	16771187.0	16742877	16742877	99.83
100020	5602326.0	5250141.8	5250141.8	93.71
100018	919081.8	117844.7	117844.7	12.82
100003	42287291.4	42135426.3	42135426.3	99.64
2	36826576.4	36763814.1	36741250.1	99.77
37	33956039.3	33870863.7	33848424.1	99.68
41	1055166.0	1596728	1050013	99.51
100016	12751128.2	12656853.1	12656853.1	99.26
1	46950644.5	46864349.7	46864349.7	99.82
100015	13725922.5	13678140.4	13672063.4	99.61
44	793419.0	1294200	775620	97.76
100017	2132046.5	2062016	2012776.8	94.41
43	804609.0	1052742	795676	98.89
12	17064577.5	16967376.7	16967376.7	99.43
100002	39441083.7	38099419.2	38091147.1	96.58

Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_xmap_stats.pl /home/bionano/access/local/jobs/10236351/output/align_final_M1 EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap2_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD.xmap step2.hybrid.cmap EXP_REFINEFINAL1_bppAdjust_cmap_Maleb_hap2_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD.xmap.stats

refContig	refContig_size	query_cov_bp	query_unique_cov_bp	query_unique_cov_percent
100007	33971428.0	33896288.8	33896288.8	99.78
41	1055166.0	0	0	0.00
43	804609.0	0	0	0.00
100002	39441083.7	39377603.9	39377603.9	99.84
100020	5602326.0	5585455.9	5585455.9	99.70
39	1178039.0	0	0	0.00
100018	919081.8	898304.4	898304.4	97.74
52	474924.0	0	0	0.00
100015	13725922.5	13672063.4	13672063.4	99.61
3	48668124.1	48690963.3	48634291.3	99.93
44	793419.0	0	0	0.00
100017	2132046.5	2123439.3	2123439.3	99.60
12	17064577.5	17020082.3	17020082.3	99.74
108	110801.0	0	0	0.00
100003	42287291.4	42242609.7	42242609.7	99.89
2	36826576.4	36516740	36516740	99.16
13	16771187.0	16742877	16742877	99.83
36	1337101.0	0	0	0.00
28	38526916.3	38500497.3	38500497.3	99.93
37	33956039.3	33857151.4	33857151.4	99.71
100016	12751128.2	12729439	12729439	99.83
15	49422970.2	48853464.2	48853464.2	98.85
1	46950644.5	46923249.5	46873563.5	99.84
100009	31504095.7	31417129	31417129	99.72

XMAP statistics calculation complete in 1 second.

Beginning archive and zip of result files...
