That is correct - even just having credit card data pass through your server without the data ever coming to rest is enough to put your server (and any attached infrastructure) into PCI scope.
Options to avoid PCI compliance difficulties:
* Use a gateway that supports a transparent redirect (for example Braintree).
* Use a gateway that send the customer off of your site (for example PayPal Express).
* Use a gateway that uses Javascript to submit directly (for example Stripe).
* Use a service like Spreedly Core that puts a transparent redirect on top of the gateway of your choice (shameless plug;
https://spreedlycore.com).
Finally, I'll caveat all of this by saying that I'm not a PCI QSA, and if you want a definitive ruling on your specific PCI case, you should retain a real QSA and ask them.
Hope this helps,