The difference in the Nginx memory size can be due to the modules you have compiled in or the amount and size of buffers you have allocated.

The difference in the PHP memory size can be anything from modules compiled in/dynamically linked to what type of scripts you're running.

One thing I can tell you with certainty is that Nginx does not affect the memory usage of PHP.