I'm not so sure anymore, after looking at the standard again. It says that it's okay to access any object via a glvalue of type char, unsigned char, or std::byte; note that this does not include arrays of the same. If you have a field of array type, and you subscript it, the expression does involve a glvalue of that array type as one of the operands - but does this constitute "access"?
A little late, but yes, subscripting an array yields a gvalue of that type, but that is not an access unless you use the glvalue in some other computation (I assume). But even if you do access it (i.e. make a bitwise copy of the entire array) that should still not be UB.
Do you have a source for this? IIRC char and std::byte have a specific aliasing exception. I.e. char* and std::byte are allowed to alias anything.
You obviously aren't allowed to modify the char or std::byte array (because that would violate the struct/class's aliasing rule).