Mortar Projects come in two flavors: Public and Private. Public Mortar Projects can be viewed and forked by anyone. Private Mortar Projects are only accessible to users in your Mortar account. When creating a new project you will need to decide if this is code that you would like to share with the world or if this is code that you want to keep within your organization.
To allow you to get started quickly we have created a base Redshift project that you can fork to create your own project. This project contains a number of runnable examples and templates. The base Mortar project is open source and hosted in Github at https://github.com/mortardata/mortar-etl-redshift.
Mortar Projects share one global namespace, and so each project must have a unique name.
mortar projects:fork email@example.com:mortardata/mortar-etl-redshift.git <your_project_name>
To read more about Mortar projects see How Mortar Projects Work.
In order to connect to Redshift this project uses psycopg2 a popular PostgresSQL adaptor for Python. This library requires your system to have python-dev and libpq-dev packages. Mortar will attempt to use this library and will generate an error if you do not have the required packages installed.
To test your project, run the following command from the root directory of your newly forked project:
This is the output if your system is all set up and ready to go:
=> mortar local:configure Installing pig-0.12 to .mortar-local... done Installing pig dependencies to .mortar-local... done Installing python to .mortar-local... done Installing user defined python dependencies... done Installing python dependencies to .mortar-local... done Installing jython to .mortar-local... done
However, if you see output like:
=> mortar local:configure Installing pig-0.12 to .mortar-local... done Installing pig dependencies to .mortar-local... done Installing python to .mortar-local... done Installing user defined python dependencies... failed ! ! Unable to setup a python environment with your dependencies, see dependency_install.log for more details
you will need to install the psycopg2 dependencies.
On Mac OS X you can use Homebrew to install PostgresSQL which will add all the dependencies necessary for psycopg2.
brew install postgresql
For alternative installation options see Installing on Mac OS X.
To install the required packages you can do:
sudo apt-get install libpq-dev python-dev
For more information see Psycopg Install Instructions.
This project is now in your Mortar account. If in the future you want to get updates made to Mortar's version of mortar-etl-redshift, use:
git fetch base git merge base/master
And that's it! You now have all of the code forked into your own Mortar project all ready to go.
Before getting into the details of our project, take a minute to set up your development environment. You should pick an editor you want to use for viewing and working with code. If you don't already have a favorite editor you can review some options here.