The 20th USCT was mustered in February 9, 1864, on Rikers, training there about a month before being sent to New Orleans region (known as the Gulf war zone or department) where its members demonstrated their bravery in the Battle for Port Hudson. They also saw duty in Texas, Florida, and Tennessee before being mustered out October 7, 1865. The 26th USCT served under the Department of the South (Union Army) in South Carolina and was very active on Johns and James Island, Honey Hill, Beaufort, and a number of other locations. Two officers and 28 enlisted men killed or mortally wounded, and 3 officers and 112 enlisted died from disease.

The 31st USCT was organized on Hart Island during April 1864 under the command of Colonel Henry C. Ward. In May 1864 the 30th Connecticut Colored Volunteers consolidated with this regiment. Additionally, a number of Black Canadians and other foreign born Blacks served in the New York regiments. From Hart Island, the 31st departed for Virginia where it was active in a number of battles, including the siege of Petersburg. The Hart Island regiment was there during the mine explosion and the fall of Petersburg on April 2, 1865. It pursued Lee's Army from April 3 through April 9 and was at Appomattox before, during and after the Confederate surrender on April 9, 1865.

The 31st USCT was one of 17 regiments that formed an advance line and moved towards Appomattox from the west and prevented Lee's army from escaping. Two Black units from other states were in this line while three other Black regiments were among other Union units positioned in the rear. A number of Confederates surrendered to those regiments and were subsequently paroled under the surrender agreement. The 31st Regiment was one of 32 regiments that made up the 25th United States Army Corps, the only Army Corps in the history of the country to be made up entirely of Black infantry regiments.

Within the 25th Army Corps, as organized in the spring of 1865 under Major General Godfrey Weitzel, the Hart Island regiment was designated part of the Third Brigade, under Colonel Edward Martindale, that in turn was part of the Corps' Second Division, under Brigadier General William Birney.

After the war the regiment was deployed to Texas and served along the Rio Grande. Three officers and 48 enlisted men were listed as killed or mortally wounded. One officer and 123 enlisted men died from disease.

About 100 Blacks entered as or subsequently became commissioned officers. These included doctors and chaplains as well as staff, line and field officers such as 20th USCT Chaplain George W. LeVere and 26th USCT Chaplain Benjamin F. Randolph. An African-American civilian James B. Schermerhorn was appointed a second lieutenant on July 22, 1864, and assigned to the 20th U.S. Colored Troops. Some other Blacks from New York, but not with the three USCTs from the state, also became officers. These included Stephen A. Swails, a first sergeant with the 54th Massachusetts (commissioned second lieutenant March 11, 1864) and Peter Vogelsang of Brooklyn who eventually became a first lieutenant.