Not sure if a normal byte will suffice, or if you need a ByteBuffer, or even a direct ByteBuffer?
Performance differences are largely application dependent, some questions to ask:
Are you going to be filling up and draining the buffer often?
Is the buffer expected to get large (1024 bytes+) before draining?
Is a smaller buffer that's emptied more often going to work better than a large buffer?
Here is a nice overview: http://www.evanjones.ca/software/java-bytebuffers.html