In The Scarlet Letter how does Hawthorne experiment with the novel? Please touch upon how he does not have clear-cut protagonists/antagonists...

In Nathaniel Hawthorne's time, the American novel was itself an experiment. One English writer, James Russell Lowell, wrote,

You steal Englishmen's books and think Englishmen's thoughts,/With their salt on  her tial your wild eagle is caught;/Your literature suits its each whisper and motion/To what will be thought of it over the ocean.

While the long prefatory essay on the Customs House is not really integral part of the narrative, it does explain the narrator's preoccupation with the story of Hester Prynne as well as his concern for the effects of the Puritan law.  For, he is a citizen with some experience in practical affairs and some interest in politics. In addition, he demonstrates his emotional response to his fellow man with the story of the Collector, "our gallant old General," and his ability to write witty, biting satire with his depiction of the Inspector.  In a sense, Hawthorne prepares his reader for his narrator's talents in the novel.

Clearly, Nathaniel Hawthorne has written a truly American novel.  For, though a bit flawed in its underdeveloped characters, it is original in its subject matter, in its strong use of symbolism, elements he carved so well that it prompted another Englishman, D.H. Lawrence, a renowed writer, to comment,

That blue-eyed darling Nathaniel knew disagreeable things in his inner soul.  He was careful to send them out in disguise.

Indeed, The Scarlet Letter is a parable in which Hawthorne scrutinizes the Puritan culture that has had such a profound impact on the American character.  Wrought with guilt because of the sins of his Puritan uncle who was a judge in the Salem Witchcraft Trials, Hawthorne (who, in his shame, changed the spelling of his surname) examines the  consequences of secret sin and of public penance, touching upon many familiar conflicts in life. To lend his parable more verisimilitude, Hawthorne incorporates real characters into his fiction such as the Reverend Mr. Wilson.  To lend his parable spiritual significance and life, Hawthorne employs symbols such as the scarlet A which changes meaning throughout the narrative and Pearl, the child who represents much more as the passionate symbol of her mother and her father's inner natures denied by Puritanism.

His ultimate message against the hypocrisy of his ancestral Puritanism is thus conveyed through the symbolism and characters:  those who acknowledge and confess their sins can be redeemed, while those who keep their sins secret, live--as Thoreau remarked--"lives of quiet desperation."  Hawthorne himself declares his novel "a tale of human fraility and sorrow."  His lesson is declared in his conclusion,

Be true! Be true!