Update: The easiest way to do this is actually to install Offset and then put a RemoveLastUserKeychains script into /usr/local/offset/logout-every (make sure it's owned by root:wheel and has 755 permissions).
This is a sequel to Deleting Mac Keychains in an Active Directory Environment, which talked about a way to delete keychains at logout using the (Apple-deprecated but still functional in Yosemite) logout hook or using a Launch Daemon that deletes all keychains at boot time.
The obvious downsides to those two methods are one being deprecated (so possibly not supported in the release after Yosemite) and the other running only at boot time (which doesn't take into account a user changing a password twice before the machine has a chance to reboot).
With a little trial and error and immense help from this post on StackExchange by a user named Haravikk, I've pieced together the exact steps to delete the user keychains at logout using a Launch Agent.
Create the shell script
We're going to create (if it doesn't already exist yet), a custom directory for your organization:
Then make the script (again, you can do this in your favorite graphical text editor, move it over to the directory, and then change ownership to root user and wheel group, but this is a fairly straightforward way to do it one fell swoop without having to change ownership later):
In the text editor, paste in:
# Add entry to a log file
echo "$(date) - Keychain deleted" >> ~/Library/Logs/NAMEOFYOURORG.log
# Delete the keychains for this user
rm -rf ~/Library/Keychains/*
trap 'onLogout' SIGINT SIGHUP SIGTERM
while true; do
sleep 86400 &
Then save (Control-X).
Make the script executable
Create the Launch Agent
You're going to create a custom .plist file
In the text editor, put in:
Save the file (Control-X) and reboot.
The Launch Agent will then run every time a user logs in, and then it'll run the shell script every time a user logs out.