Archive for development

Using ARRAY_BUFFER’s in PyOpenGL

Posted in How To, Programming, PyGLy with tags , , , , on 2013/02/16 by Adam Griffiths

I’m in the process of converting my code from Pyglet GL to PyOpenGL.
In doing so, my VBO objects stopped rendering.

It turns out the problem is glVertexAttribPointer.
The Pyglet GL version takes the last parameter (offset) as a number. I set this to 0 for arrays with no offset.

glVertexAttribPointer( in_position, 3, GL_FLOAT, GL_FALSE, 0, 0)

It seems that PyOpenGL expects the pointer value by absolute instead of relative.
The solution is to pass None instead of 0.

glVertexAttribPointer( in_position, 3, GL_FLOAT, GL_FALSE, 0, None)

If you have an actual offset to pass, you need to convert to a ctypes c_void_p (void*).
http://stackoverflow.com/questions/11132716/how-to-specify-buffer-offset-with-pyopengl

glVertexAttribPointer( in_position, 3, GL_FLOAT, GL_FALSE, 0, ctypes.c_void_p(offset))

Adobe Air hanging when using SDK but not when packaged

Posted in How To with tags , , , , , on 2013/01/30 by Adam Griffiths

I just had an issue during the development of an Adobe AIR Application.
When using Flash Pro CS 6, the application hang in certain situations for 5 odd seconds.

I found a few peculiar quirks:

  • Packaged builds didn’t have the problem.
  • Moving the project to a different directory fixed the problem.
  • Clean checkouts of the source didn’t have the problem
  • Re-creating the project in Flash Pro CS 6 didn’t fix the problem.

The console was printing the following messages

Breakpoint not set; No executable code at line ###

It turns out the problem was old breakpoints that don’t exist anymore.
There’s not trivial way to remove all the breakpoints (the button was greyed out!).
The easiest way I found (on OS-X) was to do the following:

vi ~/Library/Application\ Support/Adobe/Flash\ CS6/en_US/Configuration/Debugger/AsBreakpoints.xml

Remove all the breakpoint entries in the XML.
The file should look similar to this:

<?xml version="1.0"?>
<flash_breakpoints version="1.0">
</flash_breakpoints>

This resolved the issue.

PyGLy steps into the future

Posted in Development, PyGLy, Twisted Pair with tags , , , , , on 2012/09/21 by Adam Griffiths

I’ve been working quite heavily on PyGLy for the last few weeks and I’m incredibly pleased to announce that PyGLy is now OpenGL 3 clean!

It took more work than I hoped. Not because of PyGLy (it was already pretty good), but Pyglet’s OpenGL Core (3+) support on OS-X, is well… broken.
I had to integrate a patch written by someone else and patch out 2 of the window event handlers.
The main reason for this is that OpenGL Core on OS-X is limited to 3.2, and is Core only (no legacy compatibility).
These changes can be found in my Github repository.

Pyglet isn’t without it’s problems. It is quite heavy weight in places. There is no support for float or 1D textures.
Other problems are it’s usage of legacy calls. These are scattered throughout the code base and prevent me from using even the Label or VertexList classes.
I would LOVE to help with the development of Pyglet… but I find the code… very confusing.
It’s got a fair amount of abstraction. Tracing even a vertex buffer blows my mind.

Regardless, I hope these issues will be fixed soon.

OpenGL and GLSL support in OS-X

Posted in Twisted Pair with tags , , , , on 2012/08/13 by Adam Griffiths

I had difficulty finding the supported OpenGL and GLSL versions in OS-X.
Using Pyglet, I get errors trying to use versions GLSL 1.30 and above. Printing out the version returns 1.20.

from ctypes import *
from pyglet.gl import *

print "OpenGL version", gl_info.get_version()

plain = string_at(glGetString(GL_SHADING_LANGUAGE_VERSION)).split(' ')[0]
major, minor = map(int, plain.split('.'))
version = major*100 + minor
print "GLSL Version",version

Looking around I found this information.
Mac OS-X OpenGL Support

Legacy would infer the “Fixed Function Pipeline” and Core would be modern OpenGL.
So it seems that legacy OpenGL (normal Pyglet) is stuck with GLSL 1.20 and Core has 1.50.

This code can be found on the Pyglet source repository that enables OpenGL 3 (Core).
Enable OpenGL 3 in Pyglet

But the ouput I get is not correct.

OpenGL version 2.1 ATI-1.0.25
GLSL Version 120

According to this conversation, it appears that Pyglet ignores the OpenGL version on OS-X currently.

PyGLy Progress

Posted in Development, Twisted Pair with tags , , , on 2012/07/06 by Adam Griffiths

Work on PyGLy is progressing well. The code is better designed and cleaner than I’d hoped, so I’m very pleased.

I’m wanting to keep PyGLy thin, so not much more new functionality will be added. New features will go into higher level projects that sit on top.

I tried some NumPy optimisations with BLAS and other libs, but haven’t seen much improvement. Biggest change was running Python with ‘-O’, which got me another 10 FPS.

Dev continues!

Adventures in Kivy

Posted in How To, Programming, Rant with tags , , , , , , on 2012/03/22 by Adam Griffiths

So here is a semi-live blog as I delve into Kivy.

Mission #1: Attempt to resize the default window

Ok let’s start at the start.

We first create an object that inherits from App. Done.

Ok, and the examples all return a widget from the App’s build() method. So we don’t over-ride __init__… ok… thats… unique.

Perhaps the widget is the window if it is returned from the build() method?

Widget is not the window. Widget returned from build() is added to the window. So… where is the window created?

Window docs state the window has a constructor that lets you set the window size. Ok, cool. But we don’t create the window… so.. we can’t use the constructor… sigh.

Ok, Window has a static accessor.

kivy.core.window.Window = None

No comment on what is stored here but I assume it’s the instantiated window. It doesn’t mention when it is set… handy!

Only 1 window allowed. Ok… weird. Well… let’s try that.

import kivy.core.window.Window
AttributeError: 'module' object has no attribute 'window'

Awesome…

Let’s scour the code.

Screen capture of Kivy window module

No Window.py file……… how far does this rabbit hole go.

Ok, let’s check out the __init__.py file in the window module.

class WindowBase(EventDispatcher):
 '''WindowBase is a abstract window widget, for any window implementation.

So… window base is in there. Feel free to scream now.

So we can get this by importing kivy.core.window and just leaving the class off the end.

import kivy.core.window
...
def build(self):
    print kivy.core.window.Window.size
    return None
(800, 600)

Ok sweet, thats our window.

So let’s change its size.

def build(self):
    print kivy.core.window.Window.size
    kivy.core.window.Window.size = (1024, 768)
    print kivy.core.window.Window.size
    return None
File "/Users/adamgriffiths/Workspace/VirtualEnvs/progress_quest/lib/python2.7/site-packages/kivy/core/window/__init__.py", line 243, in _set_size
 if super(WindowBase, self)._set_size(size):
 AttributeError: 'super' object has no attribute '_set_size'

FFFFFFFUUUUUUUUUUU

Ok, so after some googling we find this post about the relationship between the App and Window classes.

His solution is to pass the size as a command line parameter……….. LAME. That kinda throws the whole configuration settings thing out the window.

Looking through the Kivy code we find this block inside their base window class’ __init__ method.

if 'fullscreen' not in kwargs:
    fullscreen = Config.get('graphics', 'fullscreen')
        if fullscreen not in ('auto', 'fake'):
        fullscreen = fullscreen.lower() in ('true', '1', 'yes', 'yup')
    kwargs['fullscreen'] = fullscreen
if 'width' not in kwargs:
    kwargs['width'] = Config.getint('graphics', 'width')
if 'height' not in kwargs:
    kwargs['height'] = Config.getint('graphics', 'height')

Ok, so we can try and set it via the global config object.

So let’s add that to our App class.

def build_config(self, config):
    config.adddefaultsection('graphics')
    config.setdefault('graphics', 'width', 1024)
    config.setdefault('graphics', 'height', 768)
    config.set('graphics', 'height', 1024)
    config.set('graphics', 'width', 768)

Run the code and……

(800, 600)

……. sigh

So let’s put some debug INSIDE kivy and see what is happening. We’ll also force it to pull the ‘graphics’ setting and ignore kwargs.

if True: #'width' not in kwargs:
    kwargs['width'] = Config.getint('graphics', 'width')
print kwargs['width']
if True: #'height' not in kwargs:
    kwargs['height'] = Config.getint('graphics', 'height')
print kwargs['height']

And the output…

800
600

…. calm blue ocean…. calm blue ocean….

Ok… google…

Hmm.. ok, this post on the kivy-users forum (because documentation is for lusers) shows a similar method.

They use the Config module directly instead of the config object passed to App….. because….. they’re not the same? Well.. let’s try it anyway.

from kivy.config import Config
Config.set('graphics', 'width', 1024)
Config.set('graphics', 'height', 768)

And our output

(800, 600)

Ok, weird because the window HAS ACTUALLY CHANGED SIZE.

Whatever. We’re all living in crazy-ville atm so let’s just pretend everything is normal.

So let’s move that inside our build_config() method.

def build_config(self, config):
    Config.set('graphics', 'height', 1024)
    Config.set('graphics', 'width', 768)

And the output

[INFO ] Kivy v1.1.1
[INFO ] [Logger ] Record log in /Users/adamgriffiths/.kivy/logs/kivy_12-03-22_40.txt
[INFO ] [Factory ] 102 symbols loaded
[INFO ] [Text ] using <pygame> as text provider
[INFO ] [Loader ] using <pygame> as thread loader
[INFO ] [Window ] using <pygame> as window provider
[WARNING] [Window ] Unable to use <pygame> as windowprovider
[CRITICAL] [Window ] Unable to find any valuable Window provider at all!
Fatal Python error: (pygame parachute) Segmentation Fault
Abort trap: 6

(╯°□°)╯︵ ┻━┻

Install GCC with XCode 4.3

Posted in How To with tags , , , , on 2012/03/21 by Adam Griffiths

XCode 4.3 removed GCC from the default installation. To get it back simply do the following:

  1. Run XCode
  2. Select XCode -> Preferences -> Downloads -> Command Line Tools -> Install

Voila! GCC is now installed and libraries such as PIL will once again install.

Installing Kivy on OS-X from PIP and Homebrew

Posted in How To, Programming with tags , , , , , on 2012/03/19 by Adam Griffiths

Begin by following this guide to get your python environment setup.

If you aren’t using virtualenv, then feel free to ignore those commands (mkvirtalenv, cdvirtualenv).

Remember to run ‘brew’ commands as the current user and not root.

Install SDL

brew install sdl sdl_image sdl_mixer sdl_ttf smpeg portmidi

Install Mercurial to gain access to PyGame repository.

brew install mercurial

Create a virtual environment to work in

mkvirtualenv kivy
cdvirtualenv

Install our Kivy dependencies and then finally, Kivy itself.

pip install cython
pip install pil
pip install hg+http://bitbucket.org/pygame/pygame</pre>
pip install kivy

If you don’t install PIL or PyGame, you will get errors such as this

(kivy-test)Vibur:kivy-test adamgriffiths$ python src/main.py 
[INFO   ] Kivy v1.1.1
[INFO   ] [Logger      ] Record log in /Users/adamgriffiths/.kivy/logs/kivy_12-03-19_4.txt
[INFO   ] [Factory     ] 102 symbols loaded
[WARNING] [Image       ] Unable to use <pygame> as loader!
[WARNING] [Image       ] Unable to use <pil> as loader!
[WARNING] [Text        ] Unable to use <pygame> as textprovider
[WARNING] [Text        ] Associated module are missing
[WARNING] [Text        ] Unable to use <pil> as textprovider
[WARNING] [Text        ] Associated module are missing
[CRITICAL] [Text        ] Unable to find any valuable Text provider at all!
 Traceback (most recent call last):
   File "src/main.py", line 2, in <module>
     from kivy.uix.button import Button
   File "/Users/adamgriffiths/Workspace/VirtualEnvs/kivy-test/lib/python2.7/site-packages/kivy/uix/button.py", line 38, in <module>
     from kivy.uix.label import Label
   File "/Users/adamgriffiths/Workspace/VirtualEnvs/kivy-test/lib/python2.7/site-packages/kivy/uix/label.py", line 94, in <module>
     from kivy.core.text import Label as CoreLabel
   File "/Users/adamgriffiths/Workspace/VirtualEnvs/kivy-test/lib/python2.7/site-packages/kivy/core/text/__init__.py", line 520, in <module>
     Label.register('DroidSans',
 AttributeError: 'NoneType' object has no attribute 'register'

Initial release of PyGLy

Posted in Development, Programming, Twisted Pair with tags , , , , on 2012/03/07 by Adam Griffiths

It’s a pretty big day for us, as I’m pushing my labor of love, PyGLy, to GitHub.

PyGLy is a 3D framework developed in pure python.

I’ve been dismayed at the state of game frameworks on Python.

There are a large number of quality 3D engines and frameworks out there. However, there are serious problems with the ‘engines’ out there that have Python bindings.

  • Not truly cross-platform (this is Python FFS!).
  • Not free.
  • Not maintained.
  • No documentation (the worst culprit).
  • Bindings are 2nd class citizens and you still need to code C/C++/Whatever.
  • Don’t work with latest versions of code.

Most engines have bindings created by the community. The problem is these are quickly dumped when the person moves on.

Python only 3D engines seem… well… stagnant.

  • PySoya and PySoy seem to be seething at each other but not really producing much.
  • PyGame is just SDL in disguise.
  • The rest… well they all 404 now.

For the most part, 3D game development on Python is dead.

So, behind the scenes, I’ve been writing my own 3D framework for Python, PyGLy.

“Framework” is an important word there. PyGLy does not force any one methodology on you. PyGLy simply provides functionality to wrap common functionality. Windows, Viewports, Scene Graph Nodes, Cameras. It’s up to you to put them together how you want.

Obviously some things are going to be coupled together. But for the most part, PyGLy just gets out of the way.

At the moment PyGLy is quite small, but it is in active development and already has features that may interest some.

I think the best case for it at the moment is for people wanting to rapidly prototype in 3D but not be abstracted from the rendering process. PyGLy lets you forget about the scene graph and just concentrate on rendering your objects. Rendering is performed via callbacks. You can make any OpenGL call you want in these callbacks.

PyGLy is the foundation of our Python 3D work, so expect it to be actively developed going forward.

The following are some of the things that we’re wanting to add in the future:

  • Shadowing.
  • Scene management (Octree, etc).
  • Cocos2D integration (CCLayer).
  • Separate OpenGL 3+ path.

As we’ve said before, Twisted Pair are true believers of Open Source, so you can find PyGLy on our GitHub repository under a very liberal license.

Animation support for CCTMXTiledMap

Posted in Development, Programming with tags , , , , , on 2012/03/07 by Adam Griffiths

I spent today learning Objective-C and working on a small module for one of our up and coming projects.

The outcome of this was CCAnimatedTMXTiledMap. A class that adds animation support to CCTMXTiledMap in Cocos2D-iphone / Kobold2D.

Twisted Pair are true believers in support Open Source and as such we’ve published the source code to our Github repository.

Python weak references to Methods and Functions

Posted in Development, How To, Programming with tags , , on 2012/03/01 by Adam Griffiths

The ‘weakref’ module in Python cannot store pointers to methods (there are exceptions but basically you can’t).

The following is a module I’ve written to bypass this. It is essentially code from the following sites, but improved by myself for storing them in containers like set([]).

References:
[1] [2]

Continue reading

Shock goes Open Source

Posted in Development, Shock, Twisted Pair with tags , , , , , , on 2011/12/04 by Adam Griffiths

Today marks a historic day for Twisted Pair, as we release the full source code to the Shock C++ Cross-Platform library.

The project files and Socket module are currently missing.

When I get time, I’ll take a day to create the project files for MSVC, Eclipse and XCode.

Look out for PyGLy, our Python graphics library, on GitHub soon.

Pyglet mouse events and rendering

Posted in Development, How To, Programming with tags , , , , on 2011/06/28 by Adam Griffiths

The Problem

I added mouse event handlers to my Pyglet application recently. I found when I moved the mouse over the window, my rendering started to create “ghost” images.

I finally found the solution hidden in the docs.

The pyglet application event loop dispatches window events (such as for mouse and keyboard input) as they occur and dispatches the on_draw event to each window after every iteration through the loop.

So every mouse move (or any other event) triggers Pyglet to call “on_draw”.

But because I’m controlling my own render loop at 60Hz, I didn’t see a need to hook into this event. So I assume Pyglet is attempting to render the scene itself, and it’s failing miserably.

This is really bad design! What if I have lots of events that are seperate? Looks like Pyglet is going to hammer the “on_draw” event.

The Solution?

Well the basic solution is to ensure you’ve hooked into the “on_draw” event and just connect it to your render call. Don’t connect it to your game update logic or you will end up updating everything.

But…

This means your frame rate will vary depending on user input, and may even end up running balls out. Which could turn a simple game into a system hog. My basic application goes from 1% (of 2 cores on a dual core system), to 20% with the mouse moving.

It also means that you need to decouple the rendering from the system update, as each event you have scheduled will trigger an “on_draw” event. If you don’t decouple them, you will render each frame twice!

The Proper Solution?

We need to over-ride the “EventLoop.idle” method as mentioned in the docs.

The code without “on_draw” looks like this:

def idle( self ):
    """An alternate idle loop than Pyglet's default.

    By default, pyglet calls on_draw after EVERY batch of events
    which without hooking into, causes ghosting
    and if we do hook into it, it means we render after every event
    which is REALLY REALLY BAD
    http://www.pyglet.org/doc/programming_guide/the_application_event_loop.html
    """
    pyglet.clock.tick( poll = True )
    # don't call on_draw
    return pyglet.clock.get_sleep_time( sleep_idle = True )

def patch_idle_loop():
    """Replaces the default Pyglet idle look with the :py:func:`idle` function in this module.
    """
    # check that the event loop has been over-ridden
    if pyglet.app.EventLoop.idle != idle:
        # over-ride the default event loop
        pyglet.app.EventLoop.idle = idle

The “on_draw” function also calls pyglet.window.flip() for us, so now we need to do that after our rendering. So add the following (obviously you need to change the window variable depending on what you’ve called it):

# switch out the default idle function for ours
patch_idle_loop()

def render( self ):
    # do render here
    # flip the buffers
    self.window.flip()

And we’re done! No more superfluous “on_draw” events and my simple application no longer spikes the CPU usage when the mouse is moved!

Registering Event Handlers in Pyglet

Posted in Development, How To, Programming with tags , , , , , on 2011/06/25 by Adam Griffiths

The Problem and Pyglet’s own solutions

Pyglet’s documentation suggests a number of methods to receive events.

The recommended method of using @ decorators works for global methods where the event object is known. But if its for a class method or the event object is passed in later, this will not work.

The alternative is to simply over-write the existing handler with something like

window.on_resize = self.on_resize

Unfortunately most of them end up with the following error:

window.set_handler( window.on_key_release, self.on_key_release )
AttributeError: 'Win32Window' object has no attribute 'on_key_release'

This discussion on the developer list explains it. This method only works when the platform’s window class has an existing method to handle the event. And not all events are handled by default, so when you come across this, BOOM!

Even then, when it works this is also a double edged sword. The existing handlers are there for a reason! For example, if you over-write the on_resize method of Window your application will no longer draw (took me a while to figure this out).

So yes, Pyglet’s documentation is really, REALLY, asking for trouble.

The Proper Solution?

Use the “push_handlers” method.

Pyglet automatically detects a method with the event’s name and dispatches the event.
So we can just add methods to our class and we’ll automatically receive the events.

class Test:
  def attachToWindow( self, window ):
    window.push_handlers( self )

  def on_key_press( self, symbol, modifiers ):
    # method name matches the 'on_key_press' event!
    print "Keypress"

Or we can manually specify which events we care about.
If we do this, we must specify the actual function that receives the event.

class Test:
  def attachToWindow( self, window ):
    window.push_handlers( on_key_press = self.on_key_press )

  def on_key_press( self, symbol, modifiers ):
    # method name matches the 'on_key_press' event!
    print "Keypress"

Grouping Voxels with Hilbert Curves

Posted in Development, Programming with tags , , on 2011/06/17 by Adam Griffiths

A friend suggested quite a good idea.

Represent Voxels internally using a Hilbert Curve, the same algorithm XKCD used to display a map of IPv4.

Once you’ve got this, you can perform an RLE compression algorithm to it.

 

This paper discusses using Octrees and Hilbert curves to represent masses of voxel data.

RLE? That’s so yesterday!

Posted in Development, How To, Programming with tags , , , on 2011/06/11 by Adam Griffiths

Before writing my RLE implementation, a friend sent me a link to the blist package for Python.

I didn’t see anything amazing on the front page. Makes a 5M size list. Sure, whatever.

What the front page fails to say is that creating that 5M size blist is insanely fast. Because it doesn’t blindly allocate memory like a normal Python list would.

The “blist” list type is faster to add / remove than a Python list, and it supports copy-on-write semantics.

So I’m actually going to remove my cryptic RLE encoding and return to normal 3D array semantics using blists!

Continue reading

Creating Voxel Volumes Quicky in Python Part 2

Posted in Development, How To, Programming with tags , , , on 2011/06/10 by Adam Griffiths

So we created a volume pretty quickly Part 1, we also implemented basic RLE and Position Aware RLE (PARLE).

Read on for a few minor improvements we can make with little effort.

Continue reading

Run Length Encoding in Python

Posted in Development, How To, Programming with tags , , , on 2011/06/10 by Adam Griffiths

The following code is my implementation of RLE in Python. I’ve included a method to modify values without having to decode the entire value.

Please note that this algorithm is inferior to the RLE with Positioning I posted about here.

Continue reading

Position Aware Run Length Encoding in Python

Posted in Development, How To, Programming with tags , , , on 2011/06/10 by Adam Griffiths

I mentioned previously wanting to improve the RLE to include position information. I’m sure it’s already got a name, but I’m going to call it Position Aware RLE (PARLE).

Below is the code I’ve written to accomplish this. I’m sure someone can come up with a more concise way of writing it, but this is the best I can do at the moment.

Continue reading

Creating Voxel Volumes Quicky in Python

Posted in Development, How To, Programming with tags , , , on 2011/06/09 by Adam Griffiths

Creating large chunks of memory is quite tricky in Python because of a number of overheads. I’ve been attempting this myself and have found a fast method to do this which combines a few handy optimisations we would need later on for a Voxel Engine anyway.

Please be aware that the code evolves along the way, so don’t blindly copy and paste the first examples =).

Also, the WordPress theme is cutting of some text. If you copy and paste it, it is still there.

This post is continued in Position Aware RLE, RLE and Part 2.

Continue reading