Over the last several weeks, the Torah has described the Mishkan (Tent of Meeting). It served as the main synagogue for the Jews and contained the Ten Commandments. When the Jews settled in Israel, they built the Holy Temple in Jerusalem for the same purpose. Today, we don't have the Holy Temple (yet!), but we do have our local shuls!
We walk quickly to shul to fulfill the verse, "Let's hurry to know G-d" (Hoshia.)
We walk slowly away from shul to show how sorry we are to leave.