Whether a product is naturally derived or not, you still have to exercise caution with it. Some of the major known irritants are natural.
In a foaming product - one person gets on fine with SLS/SLES and another one doesn't. You could get several people who react to a product, but you can't assume it's the same thing for each of them. One could be the SLS/SLES, another could be a colourant used to make it look pretty, another could be one perfume additive, and someone else could react to a different one.
It doesn't matter what the product or the "good for you" or "nasty" content, you simply can't assume that it will be ok or not ok based on someone else's experience... you still need to try for yourself.
Personally, where I have a problem with products containing SLS/SLES is mainly because you get charged a premium price for an extremely cheap ingredient... when you look at these foaming products the first ingredient will be aqua (water) and the second is invariably the foaming agent. My experience with shampoos is that my scalp itches and flakes... yet for showers I have no issue with ABC bodywash which has SLS in it... but I don't feel I'm paying a premium price for it.