Flagellated bacteria such as Escherichia coli and Bacillus subtilis exhibit effective mechanisms for swimming in fluids and exploring the surrounding environment. In isotropic fluids such as water, the bacteria change swimming direction through the run-and-tumble process. Lyotropic chromonic liquid crystals (LCLCs) have been introduced recently as an anisotropic environment in which the direction of preferred orientation, the director, guides the bacterial trajectories. In this work, we describe the behavior of bacteria B. subtilis in a homeotropic LCLC geometry, in which the director is perpendicular to the bounding plates of a shallow cell. We demonstrate that the bacteria are capable of overcoming the stabilizing elastic forces of the LCLC and swim perpendicularly to the imposed director (and parallel to the bounding plates). The effect is explained by a finite surface anchoring of the director at the bacterial body; the role of surface anchoring is analyzed by numerical simulations of a rod realigning in an otherwise uniform director field. Shear flows produced by a swimming bacterium cause director distortions around its body, as evidenced both by experiments and numerical simulations. These distortions contribute to a repulsive force that keeps the swimming bacterium at a distance of a few micrometers away from the bounding plates. The homeotropic alignment of the director imposes two different scenarios of bacterial tumbling: one with an 180° reversal of the horizontal velocity and the other with the realignment of the bacterium by two consecutive 90° turns. In the second case, the angle between the bacterial body and the imposed director changes from 90° to 0° and then back to 90°; the new direction of swimming does not correlate with the previous swimming direction.