Howdy there!

This is still very much in development, so please accept my apologies if
anything is confusing (or worse, wrong!). 
-Anna Lieb (7/3/2015)
contact me with comments/suggestions/questions: lieb.anna.m@gmail.com

/\/\/\\/\/\/\\/\/\/\\/\/\/\\/\/\/\\/\/\/\/\\
Section (0): Introduction
/\/\/\\/\/\/\\/\/\/\\/\/\/\\/\/\/\\/\/\/\/\\
	***********************
	There are two ways to run the code. Details in Sections (1) and
	(2) below.
	***********************
		(1) from command line (just compile the C++ and you're good to go)
		(2) from Python (requires you to build with cython)

	***********************
	Some things you will need:
	***********************
		--a C++ compiler. I use gcc-4.9.
		--Open MP
		--Python (I use the Anaconda distribution)
		--Cython (if you are doing option (1))
		--Jupyter notebook (latest version of iPython notebook)

	***********************
	Some things you may want:
	***********************
		--POVRAY (if you want to make fancy plots)
		--qt_tools or ffmpeg (if you want to make movies of your fancy plots)
		--EPANET (if you want to draw networks that are complicated)
		--a stiff drink (kidding) (sort of)

	***********************
	Some basics about input and output data:
	***********************
	the original code used two input files: .inp and .config
	to describe the network simulation you were running. 
	--the .inp file:
		same structure as a .inp file generated by the software EPANET. this has the
		advantage that you can use the EPANET graphical user interface to draw a big
		network and get the connectivities right. Or, if you have .inp files lying
		around from simulating them in EPANET, you can use them! Caveat: this
		code is picky about how nodes are labeled, so if you do use an
		EPANET-generated file, you need to run the python script (which lives
		in indata/) to rewrite it with acceptable numbering. To do this, just run
	
		Python cleanupinpfiles.py nameoforiginalinpfile.inp

		this generates a new file nameoforiginalinpfile2.inp which you can use for
		this code.
	--the .config file:
		not all the things the intermittency scenario needs to know about will be
	described in an EPANET inp file. So I have created a .config file which
	contains other information, such as boundary conditions, number of grid cells,
	simulation time, pressure wave speed, and number of time steps. Pretty
	self-evident, I hope.

	the output data goes to a directory called output_data, which the code
	assumes exists. If you're starting from scratch, you'll need to make this
	directory.
	
	For the python interface, I have made some helper functions to avoid
	having to deal with .inp and .config files all the time (e.g. when changing a
	parameter and rerunning the code many times.). Those functions are found in the
	module writeit.py, whose use can be examined, in, e.g. VaryLength.ipynb.


/\/\/\\/\/\/\\/\/\/\\/\/\/\\/\/\/\\/\/\/\/\\
Section (1): Calling from command line
/\/\/\\/\/\/\\/\/\/\\/\/\/\\/\/\/\\/\/\/\/\\
	
	***********************
	Why would you want use the code from command line?
	***********************
		Well, it's probably simpler and faster to compile,
		for one thing. It's also easier to add new features to the code and test them.
		Lastly, some of the features of the original code (such as writing out data to
		make into movies) has not been wrapped for calling from Python.

	***********************
	Getting started
	***********************
	--Go to the folder pipes/
		In this folder, you should have a folder called "output_data". If you don't
		have this, your simulation will crash when it tries to write output data
		there, so run 
		mkdir output_data 
		if it's not there yet. 
		If it existed already, clear out all the files so that you don't get
		old data mixed up with the stuff you're about to generate. 
	--Go to the folder called Build/
	--The compilation information is in the file called Makefile and the
		supporting info is in the file called Sysdep. Quite frankly, I think compiling
		is voodoo magic, so if it's not compiling for you, then all I have to say is
		"good luck, have fun".
	--To run a simulation, you want to use the driver routine "justrunit.cpp". so
		you would compile as follows:
		make justrunit
	--you can also compile the optimization stuff using 
		make optimizeit
		(to do optimizaiton stuff...but that cpp file may be outdated after some
		cython-related source code changes. no guarantees.)
	--calling snytax is
		./justrunit nameofinpfile.inp  nameofconfigfile.config
		(what are .inp and .config files? go back up and read about input and output
		data!)
		I have traditionally kept the .inp and .config files in the folder /indata. So,
		for example, you can run
		./justrunit 7deSeptiembre.inp ../indata/7deSeptiembre.config.
	--if your simulation is crashing, one simple debug stragegy is to crank
		down the number of time steps. Or! set the flag WTF=1, that prints a lot of random information about what's going on.
	
	***********************
	After you run a simulation		
	***********************
	You can make really pretty figures out of the data you generated in output_data (see Section (3)).


/\/\/\\/\/\/\\/\/\/\\/\/\/\\/\/\/\\/\/\/\/\\
Section (2): Calling from Python
/\/\/\\/\/\/\\/\/\/\\/\/\/\\/\/\/\\/\/\/\/\\
	***********************
	Why would you want use the code from Python?
	***********************
		It's easier to play with parameters and data visualization. You can make
		lovely figures, change parameters, dig in and look at the data in your
		networks to verify that it is what you think it is.
	***********************
	Getting started
	***********************
	You will need to use Cython to compile the C++ code into a python module. To
	do this, run

	Python setup.py build_ext -i

	There are a couple things that usually go wrong for me here:
		(i) Clang butts in and pretends to be GCC, which means that omp.h
		can't be found. I force the compiler to be gcc-4.9 so this doesn't happen. 
		(ii) The Cython bit gets confused and tries to pass Clang-specific flags
		anyway. (Are you seeing a trend here? GO AWAY, CLANG, NO ONE LIKES YOU.)
		so if you get the following error

			gcc-4.9: error: unrecognized command line option '-Wshorten-64-to-32'

		then try running the following commands in your terminal:
		
			export ARCHFLAGS="" 
			export CFLAGS = "-arch i386 -arch x86_64" 

		...Lots of other things can go wrong, dig in to the file setup.py to look
	at compilation paths and flags. I have commented out (and labeled) the lines
	that work on Orinoco so that it will compile on my macbook air. Just uncomment
	the orinco lines and comment the macbook air ones, see if it works.
	The file allthethings.pyx has all the actual wrapping for the C++ code. It's a
	bit of a mess but I did try to document all the classes you would use in a
	python call to do something interesting (e.g. run a simulation of a network or
	run an optimization.)

	once you compile
	(1) give yourself a high-five!! (it's not easy)
	(2) you can either use python scripts or notebooks to call the C++ code. I
	personally like notebooks so all the examples are in .ipynb form. But there's
	no reason you can't use a Python script to do the exact same thing if you
	like.
	(2.5) check to make sure compiled module allthethings actually works.
	Sometimes it compiles but can't load or link libraries. Or something. Which is sad and
	means...go back to beginning and figure out compiler issues. Sorry :(
	(3) there are a lot of notebooks, many correspond to examples in the paper.
	you can use them to see some of the calling syntax and possible uses. Partial
	list of them:
		--cythondemo.ipynb (just shows lots of stuff you can do with the wrapped code)
		--Convergence.ipynb (do a convergence study)
		--VaryLength.ipynb (vary length of one pipe and simulate new network.
		demonstrates using the module writeit.py to make networks without touching
		.config or .inp files)
		--mysteryBC.ipynb (optimize to fit unknown BC)
		--reallyoptimizeDH.ipynb (optimize dh/dx by controlling boundary inflow at one
		node)
	(4) You can run a notebook on your own machine just by starting one up:
	ipython noteboook
	(5) You can also run one remotely on Orinoco, which is really nice if you
	are having trouble compiling the code on your own machine (or dont' want to
	run a big simulation there). To do this, you start one ssh pipe as follows:

		ssh -N -f -L localhost:8889:localhost:8889 lieba@orinoco.dhcp.lbl.gov

	Then you start a new terminal, and ssh in normally:
	
		ssh lieba@orinoco.dhcp.lbl.gov

And once you're logged in to Orinoco, you start up a notebook with no browser,
which instead points to the pipe you already made:

	ipython notebook --no-browser --port=8889

now go to your browser (on your machine!) and enter the address

http://localhost:8889

You should see the usual ipython notebook tree. You can run cells and make
plots--Orinoco is doing the work. 

	***********************
	After you run a simulation		
	***********************
	You can sit all day and make pretty figures in matplotlib. With tex
	labels!


/\/\/\\/\/\/\\/\/\/\\/\/\/\\/\/\/\\/\/\/\/\\
Section (4): Dealing with all the data!
/\/\/\\/\/\/\\/\/\/\\/\/\/\\/\/\/\\/\/\/\/\\
After you run a command line simulation
./justrunit blah.inp blah.config

you will have filled up the folder output_data/ with a bunch of .txt and .tga
files. You should never ever have to deal with them directly!

The python script smarterputittogether.py and blockputittogether.py do the
work of writing povray scripts and running them to create images.
Smartputitogether.py can do either a topview (with false colors for pressure)
or an angleview (with pretty glass pipes!).

Notes on this:
(0) Make a folder called movie/ if it's not already there-- that's where these
scripts write the image files to.
(1) before you run either of these scripts, open them and comment out the line that says 
os.sys(command)
this prevents you from running 100 or 300 povray renderings before you know if
you like the colors, view, etc.
(2) Also check that the number of writes is correct (it should correspond to
the number of .tga files in the output_data folder)
(3) once the os.sys(command) bit is commented out, run the python script and use
povray angleview.pov
or povray topview.pov
to generate a test figure. You may want to tweak the view or color
scheme--those properties are in topview.pov and angleview.pov.
(4) when you like the test figure, uncomment the line from step (1). Run the
script, go get some coffee and put on a movie...rendering is slow!

Go to the folder movie/ and you can then stitch the together with qt_tools, for example:
quick time movies with qt_tools
	
	qt_export --sequencerate=15 tmp_000.png --dodialog test3again.mov

or with ffmpeg:
	ffmpeg -f image2 -i tmp_%03d.png -vcodec libx264 -b 2400k output-filename.mp4