iPython Notebook 18S Chimera

Started a new iPython notebook for the 18S Chimera project. Running open ref OTU picking over the weekend:

!pick_open_reference_otus.py -i /Users/hollybik/Desktop/Data/18S_chimera/chim_demux.extendedFrags_primersremoved_fastxtrimmed.fasta -o /Users/hollybik/Desktop/Data/18S_chimera/uclust_99_merged -r /macqiime/silva_111/eukaryotes_only/rep_set_euks/99_Silva_111_rep_set_euk.fasta --parallel -O 2 -s 0.1 --suppress_taxonomy_assignment --suppress_align_and_tree --prefilter_percent_id 0.0

OTU picking finished over the weekend (yay!) on my Desktop iMac for an input file that was ~2GB in size. Running taxonomy assignment next:

!assign_taxonomy.py -i /Users/hollybik/Desktop/Data/18S_chimera/uclust_99_merged/rep_set.fna -r /macqiime/silva_111/eukaryotes_only/rep_set_euks/99_Silva_111_rep_set_euk.fasta -t /macqiime/silva_111/eukaryotes_only/taxonomy_euks/99_Silva_111_taxa_map_RDP_7_levels_euks.txt -m rdp -c 0.5

Installing iPython Notebook and StarCluster

Finally taking this leap to get iPython Notebook and StarCluster up and running on my MacAir laptop. The QIIME tutorial for this process can be found here. Mainly I didn’t find the QIIME page to have enough detail, so I followed instructions on this NYU install tutorial for Mac OS X, as follows:

  • Install command line tools on Xcode (Preferences–>Downloads)
  • Checked my version of Python (yes, I have Python >2.7)
  • Installed SciPy superpack from this site (used instructions for OS X 10.8)
  • Installed PySide pyside-1.1.0-qt47-py27apple.pkg from this site
  • Installed readline, pyzmq, pygments, tornado using this command: sudo easy_install readline pyzmq pygments tornado
  • Installed iPython from Github using this command: sudo easy_install https://github.com/ipython/ipython/tarball/master
  • Tested iPython install success by typing this command: ipython notebook –pylab inline (it worked! HTML notebook launched!)
  • Followed the manual install instructions for StarCluster (see quick start install guide below) – had to download tarball, unzip and run script.
  • Followed the StarCluster config instructions (but not sure if I did this right…)

Questions I still have:

  • Do I need to install QT? Seems like I don’t need it if I’m using iPython Notebook in a web browser, but can’t find much info about possible QT dependencies.
  • Need some advice on the StarCluster config file – AWS settings and iPython plugin options
  • Then I need to figure out how to actually link iPython to AWS and start doing my analyses…

Useful Websites: