OSXBuildV2

From PyWiki

[edit] Introduction

Building Python-Ogre under OSX is a straight forward, if slightly lengthy process. However please don't expect it to be a classic UNIX/MAC build cycle as it's not. Python-Ogre is different in that it needs to build the underlying C/C++ libraries then create and compile the wrapper code. However all is not lost as there is a fairly simple build script and configuration that handles most of the complexity.

Note: we create all the libraries etc into the ./root/usr/... area so as to not overwrite any system libraries etc that you may have. Hence making the Python-Ogre build process a simple and "safe" one -- ie we never overwrite any system files or existing libraries.

[edit] What you need

an Intel Mac with OS X version 10.4.x
XCode 2.4.1

(at least that's what I am using, maybe it will work on another setup as well)

/usr/local/bin in your PATH

You 'WILL need wget and svn

[edit] Step 1

On to our Python-Ogre install by creating a master development directory under your home directory and checkout Python-Ogre from the SVN. This may take a long time, go get some coffee.

cd ~
mkdir development
cd development
svn co https://python-ogre.svn.sourceforge.net/svnroot/python-ogre/trunk/python-ogre python-ogre

[edit] Step 2

Download and build the support libraries. We use a single build script (BuildModule.py) that can retrieve, build, generate wrapper code and compile the Python-Ogre modules (do a -h for more info or look at the script). It basically reads a set of commands from the environment.py script and executes them. By default all the logging is to log.out so if you have any problems have a look there.

python python-ogre/BuildModule.py -r -b cmake

Since cmake for MacOSX is a GUI app (and the build system currently does not take that into account) we now need to manually copy the cmake executable to where it was supposed to be.

cp -f -R root/usr/CMake\ 2.6-2.app/Contents/* root/usr/

Now we can proceed with the build.

python python-ogre/BuildModule.py -r -b gccxml boost pygccxml cg scons pyplusplus

In case build fails on pygccxml: SVN might be refusing to download from pygccxml.svn.sourceforge.net because of invalid certificate. To fix this, open a new terminal and create a temporary folder. Then just start fetching pygccxml manually so SVN caches confirmation of certificate. Example:

cd /tmp
svn  co https://pygccxml.svn.sourceforge.net/svnroot/pygccxml/pygccxml_dev pygccxml
# now confirm certificate PERMANENTLY by pressing p
# afterwards, quickly press ctrl+c to cancel fetch
cd -

(Note: I encountered this problem for both pygccxml and pyplusplus, and it seemed like even when I selected accept-(p)ermanently it failed to remember the acceptance. In the end I had to edit environment.py to remove the --non-interactive option from svn, wait for the build process to stall on pygccxml, then hit (p),(enter), wait for it to stall on pyplusplus, (p), (e)nter. Bleah. Borogove 01:11, 23 September 2009 (CDT))

Installing Cg: You will need to install CG manually (may be fixed in the future). From the finder browse to the home/downloads folder and double click on the Cg-2.1_October2008.dmg file to install the CG drivers.

[edit] Step 3

Download and build the base Ogre libraries -- On the Mac we use the prebuilt ois and cegui modules (from the Ogre Dependencies) and these are installed as part of the ogre module

python python-ogre/BuildModule.py -r -b ois ogre

As of Feb 2010 ois fails to download by CVS. If you head over to the SourceForge Project you can download the tar.gz file and extract it in your development directory (making sure it's called ois, it should be already) The build script will still try to download from CVS, but upon failing, won't notice and continue on with it's build step.

If applying patch fails: As of PythonOgre SVN revision 996, applying patch for Ogre (entitled ogre_1.6.1.patch) fails. In environment.py, find first occurence of "if isMac():" and over there, change references to 1.6.3 to 1.6.1.

At this point you can run the Ogre C++ samples

cd ~/development/ogre/Mac/build/Release/

Change the plugins.cfg to look something like: (apparently I didn't need to make any changes for 1.6.1, for pythonogre svn rev 996. ivucica 2009 aug 10)

# Defines plugins to load

# Define plugin folder
# comment out as it shouldn't be defined on the Mac
# PluginFolder=/home/andy/development/root/usr/lib/OGRE

# Define D3D rendering implementation plugin
Plugin=RenderSystem_GL.so
Plugin=Plugin_ParticleFX.so
Plugin=Plugin_BSPSceneManager.so
Plugin=Plugin_OctreeSceneManager.so
Plugin=Plugin_CgProgramManager.so

Then run the demos by double-clicking the .app file in the Finder, or from the command line; for example:

./Smoke.app/Contents/MacOS/Smoke

[edit] Step 4

Generate the wrapper code(-g), compile(-c) and install it

cd ~/development
python python-ogre/BuildModule.py -g -c ogre ois
python python-ogre/BuildModule.py -b install

Note for low-end Macs: Dual Core Macs with 1GB of RAM (e.g. MacMini) are not equipped with enough memory to handle two concurrent compile processes. They simply run into memory limit too often, and apparently total CPU usage is actually lower than if you had only one process running. So you may want to force building on one core. In SVN revision 996, this is accomplished by editing PythonOgreConfig_posix.py, and defining a variable there. Punch this in: NUMBER_OF_CORES=1. Process takes about 3h on MacMini.

Tip for seeing progess: You may want to turn on output to stdout, so you can see for yourself that something is happening. Along with -g and -c, just pass additional argument -v.

Build breaking on install step?: As of revision 996, PythonOgre's setup.py refers to packages which don't appear to exist in the build (?), at least not in the 1.6.1 trunk. You can confirm this by passing -v in addition to -b at the install step: if it throws an error about a directory not existing somewhere in packages_2.5, you will want to remove the mentioned package from setup.py.

In setup.py, find a dictionary entitled PACKAGEDATA. Its member packages contains a list of packages that are to be installed. For 996's install to succeed, the following had to be commented out/removed: ogre.renderer.ogreoctreesm, ogre.renderer.ogrepaging, ogre.renderer.ogreterrain

[edit] Testing by running demos

You can now run the Python-Ogre demos

cd ./python-ogre/demos/ogre
python Demo_Smoke.py

Remember to edit the plugins.cfg file. However, as of revision 996, there was no need to do this. This file can be found in <BUILD_DIR>/python-ogre/demos.

ImportError: No module named ogre.renderer.OGRE: Edit <BUILD_DIR>/python-ogre/demos/PythonOgreConfig.py. This is entire contents of the file:

import sys
sys.path.append('../../../root/usr/lib/python2.5/site-packages/')
sys.path.append('../../../root/usr/lib')

ImportError: dlopen(<SNIP>/site-packages/ogre/renderer/OGRE/_ogre_.so, 2): Library not loaded: libboost_python-xgcc40-mt-1_38.dylib; Referenced from: <SNIP>/site-packages/ogre/renderer/OGRE/_ogre_.so; Reason: image not found

Since we have installed our packages in nonstandard path (.../root/usr/lib) we need to tell the operating system where the dynamic libraries can be found.

export OUR_BUILD_PATH=/Users/username/development/ #substitute with your build path!
 
export LD_LIBRARY_PATH=$OUR_BUILD_PATH/root/usr/lib/ #set path for .so ; this presumes your LD_LIBRARY_PATH is empty!
export DYLD_LIBRARY_PATH=$LD_LIBRARY_PATH #set path for .dylib  ; this presumes your DYLD_LIBRARY_PATH is empty!

Adding just LD_LIBRARY_PATH is not sufficient because it doesn't apply for .dylib files.

(Thanks for this last trick go to whoever wrote the 'Leopard' instructions on the bottom of the page)

[edit] Building Additional Modules

There are many (nearly 20 at time of writing) Python-Ogre modules that can be built -- and the build procedure is the same for all of them. 1. Retrieve the source - Note: You must be in the 'development' directory (one above python-ogre) when running this script. Of course you can do one module at a time and you can combine the retrieve/build/generate/compile into a single step.

cd ~/development
python python-ogre/BuildModule.py -r ode ogreode newton ogrenewt ogreal bullet

2. Build the underlying C libraries

cd ~/development
python python-ogre/BuildModule.py -b ode ogreode newton ogrenewt ogreal bullet

3. Generate the wrapper code

cd ~/development
python python-ogre/BuildModule.py -g ode ogreode newton ogrenewt ogreal bullet

4. Compile the wrapper code to make the python module

cd ~/development
python python-ogre/BuildModule.py -c ode ogreode newton ogrenewt ogreal bullet

5. And install the modules:

cd ~/development
python python-ogre/BuildModule.py -b install

[edit] Available modules

ModuleList

[edit] Troubleshooting: Fullscreen bug

I'm not sure where to put the link, so I'll do it here. As of rev. 996 and Ogre 1.6.1, there is a fullscreen bug with no keyboard and mouse events. It happens in Ogre3d C++ too. Here's some info: OSXFullscreenBug

[edit] Getting PowerPC or Universal binaries

You may need either PowerPC-only or Universal binaries. If so, keep reading. This is a work in progress and may be incomplete. This is done with revision 996. YMMV. If you already built i386-only binaries, you may need to repeat many steps mentioned above (e.g. building boost, recompiling wrapper code, ...)

[edit] 1. environment.py

find:

cls.CCFLAGS += Config.MAC_CCFLAGS
         cls.cflags += Config.MAC_cflags

after that insert

cls.LINKFLAGS += Config.MAC_LINKFLAGS

[edit] 2. PythonOgreConfig_posix.py

make changes as specified in following diff:

-   MAC_CCFLAGS = ' -D__ASSERTMACROS__ '
-   MAC_cflags = '  -D__APPLE_CC__ -DCF_OPEN_SOURCE  \
-             -I. \
-             -F/Developer/SDKs/MacOSX10.5.sdk/System/Library/Frameworks \
+   MAC_CCFLAGS = ' -arch ppc -D__ASSERTMACROS__ '
+   MAC_cflags = '  -arch ppc -D__APPLE_CC__ -DCF_OPEN_SOURCE  \
+             -I. -mmacosx-version-min=10.4 -isysroot /Developer/SDKs/MacOS10.4u.sdk \
+             -F/Developer/SDKs/MacOSX10.4u.sdk/System/Library/Frameworks \
              -DBOOST_PYTHON_NO_PY_SIGNATURES -DBOOST_PYTHON_MAX_ARITY=19  \
              -I' + os.path.join(module_dir,'code_generators','ogre') + ' ' 
+   MAC_LINKFLAGS = ' -arch ppc  -isysroot /Developer/SDKs/MacOS10.4u.sdk -syslibroot,/Developer/SDKs/MacOS10.4u.sdk '

Note that here we're building PPC only version. If you need a Universal binary, make sure you change reference to "-arch ppc" to "-arch ppc -arch i386". Note, this causes problems on Mac Mini with 1GB RAM since GCC spawns two simultaneous processes and 1GB of RAM is less than half necessary RAM for this to work. Even building single arch causes hickups on some files.

[edit] 3. Building boost

Now this gets tricky. In step 2 in main tutorial, you're compiling several tools, including boost, right? Well, now you're no longer doing it like that. (This is an enormous hack that doesn't integrate into build system.)

1. Build gccxml only.

2. Start building boost through main build system. After configure runs, abort.

3. Open /yourdevroot/boost_1_38_0/Makefile, and at the top you'll find

BJAM_CONFIG=

Change that into

BJAM_CONFIG=architecture=combined

4. Go into /yourdevroot/boost_1_38_0 if you're not already there.

5. Run make

6. Run make install

Now proceed with pygccxml, cg, and other libs as instructed in step 2.

[edit] 4. Finish the build

Continue with main tutorial after the step 2.

This should produce valid PowerPC binaries. It Worked For Us(tm).

[edit] Universal binaries

Instead of using lipo to combine the .so for different archs, or specifying both archs above, you can use a different trick. Approximately, you can do something like this:

import platform
if platform.system()=="Darwin":
    if platform.machine()=="i386":
        from ogremac_i386.renderer.OGRE import *
    else:
        from ogremac_ppc.renderer.OGRE import *
else:
    #....

Python itself is already a universal binary.

[edit] Updating the build to use python.org Python

This section is a work in progress.

Similar to above section about building universal binaries or ppc builds, let's see what to do when building with python.org's Python which installs into /Library/Frameworks instead of /System/Library/Frameworks. You want to do it in case you're packaging with py2app and your app needs to run on older OSes too.

My notes refer to doing what's necessary to do to update an already built dev folder to update to python.org's Python. Similar changes should be done when starting raw build, but since restarting from scratch is quite time consuming, I did not test that.

Working with SVN rev 996 after following main tutorial and tutorial for PowerPC/Universal binaries.

[edit] 1. Update boost_python

Suppose you're in /Users/blah/development/boost_1_38_0 as current working dir.

1. Edit your Jamroot. Instead of refering to /System/Library/Frameworks, change the reference to /Library/Frameworks.

2. Do the same in ./build/v2/tools/python.jam

3. sudo mv /System/Library/Frameworks/Python.framework /System/Library/Frameworks/Python.framework.OLD # move system python out of the way

4. make clean; make; make install # this will build and install libs in /Users/blah/root/usr/lib/

5. sudo mv /System/Library/Frameworks/Python.framework.old /System/Library/Frameworks/Python.framework # move system python back

[edit] 2. Relink with new boost_python

We have, in the end, just rebuilt ogre and ois completely, using the new boost libs.

[edit] Alternative build instructions

Building on Leopard? Try this:

This works for me, at least...

(Brief instructions; please read the standard instructions first.)

1. Download:

Note: -- I am building on a (nearly) clean install of 10.5.2 -- current svn revision is 580 (to force this use '-r 580' switch in svn command below)

   cd ~
   mkdir development
   cd development
   svn co https://python-ogre.svn.sourceforge.net/svnroot/python-ogre/trunk/python-ogre python-ogre

2. Modify:

Edit some config files in the python-ogre folder:

2a. optional: environment.py

line 507:

OPTIONALLY, if this folder already exists:

comment out:

               #[0, "mkdir ~/Library/Frameworks", ], ## Note that this will fail if the directory exists !!!

2b. Important: PythonOgreConfig_posix.py

lines 8-11:

Change to use OS X 10.5 SDK:

YOU SHOULD NOT change anything if your are using the last svn version : the build now runs fine on 10.5

NOTE:

Change the file by copying the relevant 10.4-specific line, commenting out one of the copies, and changing the other to 10.5. (The reason: We will have to change it back to 10.4 temporarily to build the ogre wrapper; this makes it easy...)

The code block should end up looking like this:

   FRAMEWORK_DIR = os.path.abspath(os.path.join(BASE_DIR, '..', 'Library','Frameworks'))   ## need this for the Mac
   # MAC_SDK_INCLUDE= '/Developer/SDKs/MacOSX10.4u.sdk/Developer/Headers/CFMCarbon' 
   MAC_SDK_INCLUDE= '/Developer/SDKs/MacOSX10.5.sdk/Developer/Headers/FlatCarbon' 
   MAC_CCFLAGS = ' -DBOOST_PYTHON_MAX_ARITY=19 -D_POSIX_C_SOURCE -DCF_OPEN_SOURCE'
   # MAC_cflags = ' --gccxml-cxxflags "-DCF_OPEN_SOURCE -D_POSIX_C_SOURCE -isysroot /Developer/SDKs/MacOSX10.4u.sdk"'
   MAC_cflags = ' --gccxml-cxxflags "-DCF_OPEN_SOURCE -D_POSIX_C_SOURCE -isysroot /Developer/SDKs/MacOSX10.5.sdk"'

2c. Important: setup.py

line 67:

NxOgre 09 seems to be broken, so:

Comment out NxOgre_09:

                       'ogre.physics.bullet','ogre.physics.PhysX','ogre.physics.NxOgre', #'ogre.physics.NxOgre_09',

3. Build:

First install support libs:

   python python-ogre/BuildModule.py -r -b -l build_libs.out.log cmake gccxml boost pygccxml cg scons

(ok)

IMPORTANT: install Cg manually if not already installed...

Then build Ogre:

   python python-ogre/BuildModule.py -g -c -l build_ogrewrapper.out.log ois ogre

(This part should run fine on 10.5 too)

4. Install:

Finally, install it:

   python python-ogre/BuildModule.py -b -l install_ogrewrapper.out.log install

To run demos:

First you will need to edit your bash ~/.profile file to include the path to the boost library (it should be in root/usr/lib, inside your python-ogre folder). From the terminal, you could use the pico editor, like this:

   pico ~/.profile

Then, this is what I did:

   #path to dynamic libraries (Standard POSIX)
   export LD_LIBRARY_PATH=$PATH:/Users/Thingy/Development/python-ogre/root/usr/lib
   
   #path to dynamic libraries (Apple-specific)
   export DYLD_LIBRARY_PATH=$LD_LIBRARY_PATH

You must also edit the plugins.cfg files in the demo folders (see main instructions) However, remove the .so extension from the plugin filenames!

-- I actually ended up editing every plugins.cfg file in the entire python-ogre folder, but I don't know if this is strictly necessary.

Then you can try:

   cd ./python-ogre/demos/ogre
   python Demo_Smoke.py

Good luck!

Retrieved from "http://wiki.python-ogre.org/index.php/OSXBuildV2"