The Scriptures indicate that the giants of Genesis 6:4 died out. Genesis 7 tells of a worldwide Flood that God sent to destroy "all flesh" upon the earth (verse 21). Since these giants were simply a natural, genetic variation of human beings, they died along with the rest of the earth's population. The only humans to escape that catastrophe were Noah and his family.
Since Noah was of a stock of smaller people, most of his descendants were of his stature. However, some of the genes to produce giants survived through the wife of Ham, one of Noah's sons. Therefore, a number of the sons of Canaan (one of Ham's sons) were giants (Numbers 13:1-2, 32-33). In Deuteronomy 2:19-21, Moses records that God destroyed the giants who dwelt in Ammon so that the children of Lot could possess the land. Those giants—who apparently descended from Canaan through a man named Anak—eventually became extinct. King Og of Bashan was the last of them to inhabit Palestine east of Jordan (Deuteronomy 3:11).
Another well-known biblical giant is Goliath of Gath (or "the Gittite"), whom the teenage David killed with a sling stone (I Samuel 17). Goliath's brother and sons were also men of great stature, and the Bible explicitly mentions that David and his men killed them all (II Samuel 21:15-22; I Chronicles 20:4-8).
Both before and after the Flood, God was directly involved in the destruction of those giant men. The reason for their destruction is not stated directly, but like Goliath, those men seemed always to be in opposition to God and to His people Israel.