Ruby code to parse and combine text files

I use this ruby code to parse several tab-delimitted text files that contain individual raters’ perceptions of a target (in this case a video). The rater id is embedded in the filename. The target video number is also embedded in the filename.

<br />#! /usr/bin/env ruby<br /><br />out = Dir.glob('*.txt')<br /><br /># open the file to write to and add the column headers<br />columns = "grouptratertmintengagetpreparetdivergetconvergetexecutetcentralizetattentivettonetactivationn" <br />"./all_ratings.txt", 'w') { |f| f.write(columns) }<br /><br />out.each do |filename|<br />  rater = filename.split('.')[0].split('_')[0]<br />  group = filename.split('.')[0].split('_')[1]  <br /> <br />  # Assign a number for the rater<br />  case rater.downcase<br />    when "rater1"<br />      rater_id = 1<br />    when "rater2"<br />      rater_id = 2<br />    when "rater3"<br />      rater_id = 3<br />    when "rater4"<br />      rater_id = 4<br />    end<br />    puts "rater: " + rater + "(#{rater_id})" + " group: " + group<br /><br />    # Open the file<br />    f =, "r").read<br /> <br />    # Split by lines - This will make sure that the end of line from Mac Classic is n<br />    str = f.gsub!(/rn?/, "n").split("n")<br /> <br />    # Identify the line number that starts the data entry for this file by finding a specific expression in the text of the file<br /> <br />    linenum = 0<br />    exp = "- Low marked by sluggishness"<br />    line = str[linenum]<br />    puts line<br />    until line.include?(exp)    <br />      line = str[linenum] <br />      linenum += 1<br />    end<br /> <br />    linenum.upto(linenum+30) do |currentline|<br />      min = (currentline-linenum)+1<br />      # add the rater_id and the group_id to the line<br />      line = group.to_s + "t" + rater_id.to_s + "t" + str[currentline] + "n"<br />"./all_ratings.txt", 'a') { |f| f.write(line) }<br />    end<br />end<br /><br />

Copy files from incrementally-numbered drives

This code moves through drives (attached via USB) that are numbered incrementally and copies the files on the drives to the local hard disk. I’m using this to more quickly pull the data off of a number of Affectiva Q-Sensors, which I connect to my computer with a USB hub.

<br />#!/bin/bash<br />for i in {1..20}<br />do<br />  # Create the directory<br />  mkdir "./sensor_data/${i}"<br />  # Check to see if the volume is mounted<br />  drive="Q${i}"<br />  if mount|grep $drive;<br />  then<br />    echo "${drive} is mounted"<br />    # move the files over to the directory<br />    cp -r /Volumes/${drive}/ ./sensor_data/${i}/<br />  else<br />    echo "${drive} is NOT mounted"<br />  fi<br />done<br /><br /><br /><br /><br />

Create a filled line graph with R

I used the code below to create a presentation-quality graph of data on how individual’s activation level (measured as electrodermal activity) changes over time during a group-based task. (Click on the image to enlarge.)

<br />quartz(width=16,height=8,type="pdf",file="indiv_eda_z.pdf",dpi=600)<br />par(xpd=TRUE)<br />par(family="Tahoma", bg="white", mar=c(3,3,3,3), mgp=c(1,1,1))<br /><br />ylim <- c(-1.5, 1.5)<br />xlim <- c(-540, 2700)<br />x <- aggsub$task_time<br />y <- aggsub$eda_z<br />ylo <- rep(min(y), length(y))<br />plot(x,y, type="n", ylim=ylim, axes=FALSE, ylab="Electrodermal Activity", xlab="Time (in minutes)", xlim=xlim, col="dark green")<br /><br />xpos <- seq(-540, 2700, 180)<br />lab <- seq(-9, 45, 3)<br />axis(1, at=xpos, labels=lab, cex =1.5, lwd=.5, lty=3, tck=1, col="dark gray", pos=-1.5, col.axis="dark gray")<br /><br />ypos <- seq(-1.5, 1.5, .5)<br />axis(2, at=ypos , labels=ypos, cex =1.5, las=2, tck=1, lwd=.5, lty=3, col="dark gray", pos=-540, col.axis="dark gray")<br /><br />zerox <- -540:2700<br />zeroy <- rep(0, length(zerox))<br />lines(zerox, zeroy, lty=1, lwd=2, col="red")<br /><br /><br />lines(x,y, lwd=2.5,col="dark green")<br />xx <- c(x, rev(x))<br />yy <- c(ylo, rev(y))   <br />polygon(xx, yy, col="light green", border=FALSE)<br /><br />sessionstart <- min(x)<br />taskstart <- 0<br />taskend <- 1800<br />recordend <- 1920<br />sessionend <- max(x)<br /><br />polygon(c(sessionstart, sessionstart, taskstart, taskstart), c(1.5, min(y), min(y), 1.5), col="#0015FF25", border=FALSE)<br />text(-270, 1.25, "Pre-Task Survey", col="dark blue")<br /><br /><br />#polygon(c(taskstart, taskstart, taskend, taskend), c(1.5, min(y), min(y), 1.5), col="#EAFF0025", border=FALSE)<br />text(900, 1.25, "Group Members Work to Develop Recruitment Video", col="dark green")<br /><br /><br />polygon(c(taskend, taskend, recordend, recordend), c(1.5, min(y), min(y), 1.5), col="#FA050535", border=FALSE)<br />text(recordend-(recordend-taskend)/2, 1.25, "RecordnVideo", col="red")<br /><br /><br />polygon(c(recordend, recordend, sessionend, sessionend), c(1.5, min(y), min(y), 1.5), col="#0015FF25", border=FALSE)<br />text(sessionend-(sessionend-recordend)/2, 1.25, "Post-Task Survey", col="dark blue")<br /><br /><br />

Create animated graphs with R

I collected some longitudinal data on how group members’ electrodermal activity changes over the course of a group task. Displaying the data in motion was helpful for emphasizing a few points, which I won’t re-hash here. The size of the bubbles in the graph below correspond to the average amount of variation within groups in the sample.

Click on the GIF file below to see the result, which uses the animation package in R to create a moving graph.

 # Record this as a movie  
ymin <- -1.5
ymax <- 1.5
xmin <- -7
xmax <- 45
aggsub <- agg[agg$min1 >= (xmin+2) & agg$min1 <=xmax,]
sessionstart <- xmin
taskstart <- 0
taskend <- 30
recordend <- 32
sessionend <- xmax
for (i in (xmin+2):xmax) {
x <- aggsub[aggsub$min1 >= xmin & aggsub$min1 <=i, c("min1")]
y <- aggsub[aggsub$min1 >= xmin & aggsub$min1 <=i, c("eda_z")]
r <- aggsub[aggsub$min1 >= xmin & aggsub$min1 <=i, c("eda_z_sd")]
plot(x,y,ylim = c(-1.5,1.5), xlim=c(xmin,xmax),ylab="Electrodermal Activity", xlab="Time", axes=FALSE, type="n")
xpos <- seq(xmin, xmax, 3)
lab <- seq(xmin, xmax, 3)
axis(1, at=xpos, labels=lab, cex =1.5, lwd=.5, lty=3, tck=1, col="dark gray", pos=ymin, col.axis="dark gray")
ypos <- seq(ymin, ymax, .5)
axis(2, at=ypos , labels=ypos, cex =1.5, las=2, tck=1, lwd=.5, lty=3, col="dark gray", pos=xmin, col.axis="dark gray")
zerox <- xmin:xmax
zeroy <- rep(0, length(zerox))
lines(zerox, zeroy, lty=1, lwd=2, col="red")
polygon(c(sessionstart, sessionstart, taskstart, taskstart), c(ymax, ymin, ymin, ymax), col="#0015FF25", border=FALSE)
text(taskstart-(taskstart-xmin)/2, 1.35, "Pre-Task Survey", col="dark blue")
if(i >= taskstart) {
#polygon(c(taskstart, taskstart, taskend, taskend), c(1.5, min(y), min(y), 1.5), col="#EAFF0025", border=FALSE)
text(taskend-(taskend-taskstart)/2, 1.35, "Group Members Work to Develop Recruitment Video", col="dark green")
if(i >=taskend) {
polygon(c(taskend, taskend, recordend, recordend), c(ymax, ymin, ymin, ymax), col="#FA050535", border=FALSE)
text(recordend-(recordend-taskend)/2, 1.35, "RecordnVideo", col="red")
if(i >=recordend) {
polygon(c(recordend, recordend, sessionend, sessionend), c(ymax, ymin, ymin, ymax), col="#0015FF25", border=FALSE)
text(sessionend-(sessionend-recordend)/2, 1.35, "Post-Task Survey", col="dark blue")
symbols(x, y, circles=r, inches=.5, fg="white", bg="dark blue", add=TRUE)
lines(x,y, lwd=1,col="light gray", lty=3)
}, interval=.25, moviename="eda_z_team_full.gif", loop=1, filename="eda_z_team_full", fileext="png", outdir=getwd(), width=1600, height=800)

Combine PDFs generated by Word when a document has multiple layouts

Sometimes Mac OS X’s quartz and Microsoft’s Word don’t play so well together. For example, when a single Word document mixes layouts (e.g., portrait, landscape), printing the document yields multiple PDF files — one for each section of the document where there is a layout switch. Here’s the workflow I’m currently using to streamline the process of creating a PDF from a Word document with multiple layouts.


  1. If you don’t already have it, download and install Quicksilver
  2. Create an applescript file using this code (to do so, copy and paste into a text editor, then save as something like Combine PDFs.scpt)
  3. Move the script you just created to ~/Library/Application Support/Quicksilver/Actions/


  1. Save a Word document as PDF
  2. Highlight the multiple PDF files that are created if the document has multiple layouts
  3. Invoke Quicksilver (usually Control + Spacebar)
  4. Command + G to grab the files from Quicksilver
  5. Tab to the second pane and type “Combine PDFs”
  6. Press enter
  7. Quicksilver will re-invoke with the new file
  8. Tab to the next pane and type “rename”
  9. Give the file your desired name

Batch convert videos and reduce filesize

*Must have ffmpeg to run the code below

#!/bin/bash<br />for i in *.MPG<br />do <br />    ffmpeg -i ${i/.*/}.MPG -qscale 2.5 ${i/.*/}.mp4<br />    # ffmpeg -i $i.MPG -sameq $i.mp4<br />done<br />

Using WinBUGS with R on a Mac

<br />require(MCMCpack)<br />require(arm)<br />p1<-bugs(, p.inits, p.parms, "{MODEL-FILENAME}.bug", n.chains=3, n.iter=5000,"/Applications/WinBUGS14", useWINE=TRUE,"{WORKING-DIRECTORY}", debug=TRUE)<br />

Connect R to a MAMP MySQL Database

# NOTE: Edited since originally published due to a change in R...<br />require(RMySQL)<br />con &lt;- dbConnect(MySQL(), username="{USERNAME}", host="localhost", dbname="{DATABASE}", password="{PASSWORD}", unix.sock="/Applications/MAMP/tmp/mysql/mysql.sock")<br />res <- code="" con="" dat="" dbsendquery="" fetch="" from="" res=""><br /></->