Many of Emily Dickinson's poems can be seen as a struggle between two senses of "self." These two "selves", "the social self" and "the poetic self" are present in poem #670, "One need not be a Chamber - To be Haunted." In this case, the primary "self" is "the social self." This poem is the description of the struggle "the social self" has with "the poetic self."
The corridors of the brain are being compared to the vastness of an empty house, haunted by a ghost. In this poem, the primary self is "the social self" and he ghost is "the poetic self." The relationship between the two exists internally.
"The social self" feels a need to be accepted by society. It is aware of what is considered to be normal and works hard to maintain that image. Unfortunately, the internal presence of "the poetic self" is working to counter the efforts of "the social self." "The poetic self" thinks things and does things that are not generally accepted by society. By being haunted by her "self", "the social self" is unable to escape from the haunting.
The second stanza of the poem reads, "Far safer, of a Midnight Meeting - Extended Ghost - Than its interior confronting - That Cooler Host" (Dickinson). "The social self" is obviously more afraid of this internal haunting than it would be by the actual presence of a ghost in the house.
The third stanza furthers the claim that "the social self' would rather have an actual person or ghost chasing them than an internal opposition impossible to escape. It also explains the loneliness that exists behind closed doors, with the presence of the negative, disturbed "poetic self."
The fourth stanza is even more disturbing ...