Williams (who was sometimes billed as Spencer Williams Jr.) was born in Vidalia, Louisiana, where the family lived on Magnolia Street. As a youngster, he attended Wards Academy in Natchez, Mississippi. He moved to New York City when he was a teenager and secured work as call boy for the theatrical impresario Oscar Hammerstein. During this period, he received mentoring as a comedian from the African American vaudeville star Bert Williams. Williams began his studies at the University of Minnesota, taking some time out to serve his country.
Williams served in the U.S. Army during and after World War I, where he rose to the rank of 1st Sergeant. During the course of his time in service, Williams traveled the world, serving as General Pershing's bugler while in Mexico before he was promoted to camp 1st sergeant. In 1917, Williams was sent to France to do intelligence work there. After World War I, Williams continued his military career; he was part of a unit whose job was to create war plans for the Southwestern United States, in case they might ever be needed. He arrived in Hollywood in 1923 and his involvement with films began by assisting with work on Octavus Roy Cohen material. Williams began to snag bit roles in motion pictures, including a part in the 1928 Buster Keaton film Steamboat Bill, Jr. Though he had found steady work since arriving in California, Williams had a short period in 1926 where there were no roles for him; he then went to work as an immigration officer. In 1927, Williams was working for the First National Studio, going on location to Topaz, Arizona to shoot footage for a film called The River.
In 1929, Williams was hired by producer Al Christie to create the dialogue for a series of two-reel comedy films featuring all-black casts. As Williams began to gain the trust of Christie, he was eventually appointed the responsibility to create The Melancholy Dame. This film is considered the first black talkie. Due to the pressures of the depression coupled with the lowering demand for black short films, Williams and Christie separated ways. Williams struggled for employment during the years of the Depression and would only occasionally be cast in small roles. Movies included a brief appearance in Warner Brothers blockbuster film Public Enemy in 1931 in which he was uncredited. The films, which played on racial stereotypes and used grammatically tortured dialogue, included The Framing of the Shrew, The Lady Fare, Melancholy Dame, (first Paramount Studios all African-American cast "talkie"), Music Hath Charms, and Oft in the Silly Night. Williams wore many hats at Christie's; he was a sound technician, wrote many of the scripts and was assistant director for many of the films. He was also hired to cast African-Americans for Gloria Swanson's Queen Kelly and produced the silent film Hot Biscuits in the same year. Williams also did some work for Columbia as the supervisor of their Africa Speaks recordings. Williams was also active in theater productions, taking a role in the all African-American version of Lulu Belle in 1929.
By 1931, Williams and a partner had founded their own movie and newsreel company called the Lincoln Talking Pictures Company. The company was self-financed.[ Williams, who had experience in sound technology, built the equipment, including a sound truck, for his new venture.
Prior to his involvement with Amos 'n' Andy, Williams was immensely popular among the African-American audiences. U.S. radio comedians Freeman Gosden and Charles Correll, who cast Williams as Andy, were able to claim that they were the ones who found Williams and gave him the chance to be seen in the limelight because he was virtually unknown amongst the white audience.
In 1948, Gosden and Correll were planning to take their long-running comedy program Amos 'n Andy to television. The program focused on the misadventures of a group of African Americans in the Harlem section of New York City. Gosden and Correll were white, but played the black lead characters using racially stereotypical speech patterns. They had previously played the roles in blackface make-up for the 1930 film Check and Double Check, but the television version used an African American cast.
Gosden and Correll conducted an extensive national talent search to cast the television version of Amos 'n Andy. News of the search reached Tulsa, where Williams was sought out by a local radio station that was aware of his previous work in race films. A Catholic priest, who was a radio listener and a friend, was the key to the whereabouts of Williams. He was working in Tulsa as the head of a vocational school for veterans when the casting call went out. Williams successfully auditioned for Gosden and Correll, and he was cast as Andrew H. Brown. Williams was joined in the cast by New York theater actor Alvin Childress, who was cast as Amos, and vaudeville comedian Tim Moore, who was cast as their friend George "Kingfish" Stevens. When Williams accepted the role of Andy, he returned to a familiar location; the CBS studios were built on the former site of the Christie Studios.Until Amos 'n' Andy, Williams had never worked in television. Amos 'n Andy was the first U.S. television program with an all-black cast, running for 78 episodes on CBS from 1951 to 1953.However, the program created considerable controversy, with the NAACP going to federal court to achieve an injunction to halt its premiere. In August 1953, after the program had recently left the air, there were plans to turn it into a vaudeville act with Williams, Moore and Childress reprising their television roles. It is not known if there were any performances. After the show completed its network run, CBS syndicated Amos 'n Andy to local U.S. television stations and sold the program to television networks in other countries. The program was eventually pulled from release in 1966, under pressure from civil rights groups that stated it offered a negatively distorted view of African American life. The show would not be seen on nationwide television again until 2012.
While the show was still in production, Williams and Freeman Gosden clashed over the portrayal of Andy, with Gosden telling Williams he knew how Amos 'n' Andy were meant to talk. Gosden never visited the set again.
Williams, along with television show cast members Tim Moore, Alvin Childress, and Lillian Randolph and her choir, began a US tour as "The TV Stars of Amos 'n' Andy" in 1956. CBS considered this a violation of their exclusivity rights for the show and its characters; the tour came to a premature end. Williams, Moore, Childress and Johnny Lee, performed a one-night show in Windsor, Ontario in 1957, apparently without any legal action being taken.
Williams returned to work in stage productions. In 1958, he had a role in the Los Angeles production of Simply Heavenly; the play had a successful New York run. His last credited role was as a hospital orderly in the 1962 Italian horror production L'Orribile Segreto del Dottor Hitchcock.
After his failed attempts to find success in the film industry once again, Williams decided to fully retire and began to live off of his pension that he was receiving from his time with the US Military.
Williams died of a kidney ailment on December 13, 1969, at the Sawtelle Veterans Administration Hospital in Los Angeles, California. He was survived by his wife, Eula. At the time of his death, news coverage focused solely on his work as a television actor, since few white filmgoers knew of his race films. The New York Times obituary for Williams cited Amos 'n Andy but made no mention of his work as a film director. A World War I veteran, he is buried at Los Angeles National Cemetery.
When friends and family from Vidalia, Louisiana were interviewed for a local newspaper article in 2001, he was remembered as a happy person, who was always singing or whistling and telling jokes. His younger cousins also recalled his generosity with them for "candy money"; just as he was seen on television as Andy, he always had his cigar. On March 31, 2010, the state of Louisiana voted to honor Williams and musician Will Haney, also from Vidalia, in a celebration on May 22 of that year.