Why don't engines allow concave shapes?

Short answer: Math

Long answer: Calculating collisions with concave shapes is exponentially more complex in term of mathematical calculations than Convex shapes. This is true in game engines and just physics math in general.

Some of the answers to this thread add further detail. https://www.reddit.com/r/gamedev/comments/64uju9/can_someone_eli5_why_all_the_rigid_body_physics/

3 Likes