Gene Ellison was born in Sylacauga where he grew up playing Little League baseball. He played football in junior high school and Tom Calvin, former Pittsburgh Steeler great, was the Sylacauga head football coach at the time. During his high school days at Sylacauga, Ellison played football, basketball and ran track.

By the time he was a senior, he was captain of all three sports. On the varsity football team Ellison was a running back on offense and defensive back on defense. The Aggies were 7-3 during his sophomore season, 6-2-2 his junior year and 7-2-1 his senior year. He would graduate from high school in 1961.

Although Ellison was not recruited after graduating from Sylacauga High School, he wanted to continue playing football. He went to North Alabama looking to walk on.

Ellison was cut after only three days of practice. He decided to go to Livingston University and try out for that football program. In 1961 he earned a spot on the football team as a freshman.

During his sophomore year in 1962, Ellison had the opportunity to play against the team that did not offer him a scholarship. Late in the game and trailing 6-0, Ellison ran back a punt for a touchdown. The game ended in a 6-6 tie. He received a four year scholarship at Livingston for his efforts.

Ellison made All-Conference his sophomore and senior years and made second team All-Conference his junior year. He was captain of the football team his final two years. Ellison graduated from Livingston in 1965 with B.S. degree in Education.

Ellison landed his first teaching and coaching job at Choctaw County High School in Butler in 1965. He was there one year as the offensive line coach for football, head basketball coach and head track coach. He moved to Woodlawn High School in Birmingham for the next three years as an assistant football coach.

In the fall of 1970, Huffman fielded its very first football team and Ellison was named as the offensive coordinator. In his fourth season, Huffman played for the State Championship losing to Banks in the title game. Ellison became the defensive coordinator the fifth and final year he was with the Vikings.

In 1974, Ellison took over the Pell City Panther football program, his first head coaching assignment. That first season was an outstanding one as Ellison led the Panthers to a 10-3 record and lost 6-0 in the semi-finals of the playoffs against Atmore, who won the state title the next week.

He started the varsity girls basketball program and was the very first head coach of the Lady Panthers in 1975. Ellison coached three more seasons at Pell City but failed to make it to the playoffs. He finished his five year run at Pell City with a record of 29-25.

Ellison made the move to Mountain Brook in 1979 where he was the defensive coordinator the first year before taking over the head coach duties the next season. He ended up being head coach of the Spartans for 12 seasons and made it to the state playoffs four of those years.

After 27 years of coaching, Ellison moved to Florida and began coaching again at Woodham High School in Pensacola. After three years of coaching in Florida, Ellison had a total of 30 years coaching experience and retired. Following a year in business, Ellison became a head football coach again in Florida at Northview High School near Pensacola.

After three years he moved to Crestview High School. Ellison stayed there two years and finally retired after 35 years of coaching. He was head coach 25 years and an assistant coach 10 years.