Just noticed something strange. In making Graphics3D, when I add Opacity as option to Polygon, it slows down things so much. When I remove the Opacity, it speeds up again.
{(*Opacity[0.2],*)Polygon[{{-4, -4, 0}, {4, -4, 0}, {4, 4, 0}, {-4, 4, 0}, {-4, -4, 0}}]},