Final Aquarium Project data now ready!

Got the demultiplexed data back from all 4 Aquarium project runs, and now we’re going forward with OTU picking. Starting off with closed ref, to see what we get:

pick_closed_reference_otus.py -i /Users/hollybik/Desktop/Data/aquarium_project/All4MiseqRuns_demult_30Apr14/AQUEXP_all_MPF_2.fasta -o /Users/hollybik/Desktop/Data/aquarium_project/All4MiseqRuns_demult_30Apr14/uclust_closedref_30Apr14 -r /macqiime/greengenes/gg_13_8_otus/rep_set/97_otus.fasta -t /macqiime/greengenes/gg_13_8_otus/taxonomy/97_otu_taxonomy.txt --parallel -O 2
Advertisements

Aquarium data – Round 2

Guillaume re-demultiplexed the Aquarium data for us – we found one barcode that was used twice (so had to throw out those samples, 42.SedimentCoralpond.2 and 0.WaterFreshwater.2). Newest demultiplexed data allows 2 mismatches per merged dual index long barcode (so 1 per mate barcode). Concatenated data on Merlot and downloaded to my iMac:

cat /share/eisen-z2/gjospin/Slims_Eisenlab_backup/AQUEXP1/demul_9_reverse/qiime_ready/AQUEXP1.MPF.*.faa > AQUEXP1.MPF.all_reads_merged_20Jun.faa
cat /share/eisen-z2/gjospin/Slims_Eisenlab_backup/AQUEXP1/demul_9_reverse/qiime_ready/AQUEXP1.MPR.*.faa > AQUEXP1.MPR.all_reads_merged_20Jun.faa
cat /share/eisen-z2/gjospin/Slims_Eisenlab_backup/AQUEXP1/demul_9_reverse/qiime_ready/AQUEXP1.M.*.faa > AQUEXP1.M.all_reads_merged_20Jun.faa

And then kicked off the QIIME analysis (uclust open reference OTU picking, with parameters file enabling reverse strand match):

pick_open_reference_otus.py -i /Users/hollybik/Desktop/Data/aquarium_project/demul_trim_prep_2mismatcheslongBC/AQUEXP1.MPF.all_reads_merged_20Jun.faa -r /macqiime/greengenes/gg_12_10_otus/rep_set/99_otus.fasta -o /Users/hollybik/Desktop/Data/aquarium_project/demul_trim_prep_2mismatcheslongBC/uclust_openref_20Jun --parallel -O 2 -p /Users/hollybik/Desktop/Data/aquarium_project/mapping_files/aquarium_parametersfile.txt

Continuation 6/28/13 – Next filtered out the Koi pond samples, and any samples that had less than 1000 observations (sequences):

filter_samples_from_otu_table.py -m /Users/hollybik/Desktop/Data/aquarium_project/mapping_files/AquariumSampleMap8.txt -n 1000 -o /Users/hollybik/Desktop/Data/aquarium_project/demul_trim_prep_2mismatcheslongBC/uclust_openref_20Jun/filtered_otu_tables/otu_table_mc2_w_tax_no_pynast_failures_filtered1000seqs_noKoiPond.biom -i /Users/hollybik/Desktop/Data/aquarium_project/demul_trim_prep_2mismatcheslongBC/uclust_openref_20Jun/otu_table_mc2_w_tax_no_pynast_failures.biom --sample_id_fp /Users/hollybik/Desktop/Data/aquarium_project/mapping_files/samples_to_keep_noKoi.txt

Removed genus Brachybacterium from samples:

filter_taxa_from_otu_table.py -i /Users/hollybik/Desktop/Data/aquarium_project/demul_trim_prep_2mismatcheslongBC/uclust_openref_20Jun/filtered_otu_tables/otu_table_mc2_w_tax_no_pynast_failures_filtered1000seqs_noKoiPond.biom -o /Users/hollybik/Desktop/Data/aquarium_project/demul_trim_prep_2mismatcheslongBC/uclust_openref_20Jun/filtered_otu_tables/otu_table_mc2_w_tax_no_pynast_failures_filtered1000seqs_noKoiPond_noBrachybacterium.biom -n g__Brachybacterium

Filtered out “dead OTUs” (no sequences left after filtering):

filter_otus_from_otu_table.py -i /Users/hollybik/Desktop/Data/aquarium_project/demul_trim_prep_2mismatcheslongBC/uclust_openref_20Jun/filtered_otu_tables/otu_table_mc2_w_tax_no_pynast_failures_filtered1000seqs_noKoiPond_noBrachybacterium.biom -o /Users/hollybik/Desktop/Data/aquarium_project/demul_trim_prep_2mismatcheslongBC/uclust_openref_20Jun/filtered_otu_tables/otu_table_mc2_w_tax_no_pynast_failures_filtered1000seqs_noKoiPond_noBrachybacterium_nodeadOTUs.biom -n 1

Then running the core_diversity_analyses.py script:

core_diversity_analyses.py -i /Users/hollybik/Desktop/Data/aquarium_project/demul_trim_prep_2mismatcheslongBC/uclust_openref_20Jun/filtered_otu_tables/otu_table_mc2_w_tax_no_pynast_failures_filtered1000seqs_noKoiPond_noBrachybacterium_nodeadOTUs.biom -o /Users/hollybik/Desktop/Data/aquarium_project/demul_trim_prep_2mismatcheslongBC/uclust_openref_20Jun/core_diversity_analyses -m /Users/hollybik/Desktop/Data/aquarium_project/mapping_files/AquariumSampleMap8.txt -e 90 -t /Users/hollybik/Desktop/Data/aquarium_project/demul_trim_prep_2mismatcheslongBC/uclust_openref_20Jun/rep_set.tre -c SampleType,Location,SampleTypeLocation,Days

Getting started with Aquarium Project data

The data is in – time to get started in QIIME! Walked David through some workflows, but then executed my own reference-based OTU picking as a first pass through the data. Note: 4 parallel OTU picking jobs was too much for my iMac – suggest dropping this to 2 jobs next time so my computer doesn’t slow to a crawl.

Commands I ran:

parallel_pick_otus_uclust_ref.py -i /Users/hollybik/Desktop/Data/aquarium_project/demul_trim_prep_noBCmismatches/input_data/AQUEXP1.MPF.all_reads_merged_30May.faa -o ~/Desktop/Data/aquarium_project/demul_trim_prep_noBCmismatches/uclust_ref -r /macqiime/greengenes/gg_12_10_otus/rep_set/99_otus.fasta -O 4 −−enable_rev_strand_match
 
pick_rep_set.py -i /Users/hollybik/Desktop/Data/aquarium_project/demul_trim_prep_noBCmismatches/uclust_ref/AQUEXP1.MPF.all_reads_merged_30May_otus.txt -r /macqiime/greengenes/gg_12_10_otus/rep_set/99_otus.fasta -l pick_rep_set.log
 
assign_taxonomy.py -i /Users/hollybik/Desktop/Data/aquarium_project/demul_trim_prep_noBCmismatches/uclust_ref/AQUEXP1.MPF.all_read_merged_30May_rep_set.fasta -m rdp -r /macqiime/greengenes/gg_12_10_otus/rep_set/99_otus.fasta -t /macqiime/greengenes/gg_12_10_otus/taxonomy/99_otu_taxonomy.txt
 
make_otu_table.py -i /Users/hollybik/Desktop/Data/aquarium_project/demul_trim_prep_noBCmismatches/uclust_ref/AQUEXP1.MPF.all_reads_merged_30May_otus.txt -o AQUEXP1.MPF.all_reads_merged_30May_otu_table.biom -t /Users/hollybik/Desktop/Data/aquarium_project/demul_trim_prep_noBCmismatches/uclust_ref/rdp_assigned_taxonomy/AQUEXP1.MPF.all_read_merged_30May_rep_set_tax_assignments.txt