The anatta mental state is a stage of cultivation where the mind is in a tranquil and calm state, completely free of the constraints caused by ones surroundings. Anatta is the Buddhist concept of “non-self” (more information).
During the Lord of Nether's Disciple Selection the test on the eighth level was to reach this state of cultivation while within the Black Infernal Tower. Within the tower was powerful black flames that could cripple ones cultivation and extremely powerful Demon Beasts chained up, making it more difficult to concentrate.
During Venerable Redsoul class, he taught to create Spiritual Flames. In order to create a Spiritual Flame, the Soul Realm must reach the Anatta Mental State, then gather their intent onto their palm.