Loomis commanded the 26th Illinois Infantry during the Civil War and held the rank of colonel for three years. He and his Regiment participated in most of the important battles in the Western Theater. During 1862 the regiment was attached to the 3rd Division and then the 2nd Division of the Army of the Mississippi. In 1863 the regiment formed a part of the 16th, then the 17th Corps of the Army of the Tennessee. For the remaining years of the War, the regiment was attached to the 15th Army Corps. At the time of the Vicksburg Campaign, John Mason Loomis was in command of the 1st Brigade, 1st Division of the 16th Corps, consisting of the 26th Illinois, 90th Illinois, 12th Indiana, and the 100th Indiana. The 16th Corps was commanded by Major General William Tecumseh Sherman from Vicksburg to Jackson.
