First, press F4 so you get the options for the debugging mode (F5-F8). Now reconfigure "Basic" (should be F5 - I am writing this out of my head) so it only shows the grid. I also activated the FPS. If you press F5 in a running game now, you got a grid which shows you every tile and every chunk (a chunk is 32x32 tiles if I remember correctly).
Hope that helps some of you! I would have been glad if someone told me earlier at least
Here is an old screenshot I found to show you what I'm talking about: