Not all pagans. Seven specific nations.
Deuteronomy 7: ( emphasis mine )
7:1
ืื ืืืืื ืืืื ืืืืื ืืึพืืืจืฅ ืืฉืจึพืืชื ืืึพืฉืื ืืจืฉืชื ืื ืฉื ืืืืึพืจืืื ืืคื ืื ืืืชื ืืืืจืืฉื ืืืืืจื ืืืื ืขื ื ืืืคืจืื ืืืืื ืืืืืืกื ืฉืืขื ืืืื ืจืืื ืืขืฆืืืื ืืืื
When the Lord your God shall bring you into the land which you are entering to possess, and has cast out many nations before you, the Hittites, and the Girgashites, and the Amorites, and the Canaanites, and the Perizzites, and the Hivites, and the Jebusites, seven nations greater and mightier than you;
7:2
ืื ืชื ื ืืืื ืืืืื ืืคื ืื ืืืืืชื ืืืจื ืชืืจืื ืืชื ืืึพืชืืจืช ืืื ืืจืืช ืืื ืชืื ืื
And when the Lord your God shall deliver them before you; you shall strike them, and completely destroy them; you shall make no covenant with them, nor show mercy to them;
7:3
ืืื ืชืชืืชื ืื ืืชื ืืึพืชืชื ืืื ื ืืืชื ืืึพืชืงื ืืื ืื
And you shall not make marriages with them; your daughter you shall not give to his son, nor his daughter shall you take to your son.
7:4
ืืึพืืกืืจ ืืชึพืื ื ืืืืจื ืืขืืื ืืืืื ืืืจืื ืืืจื ืืฃึพืืืื ืืื ืืืฉืืืื ืืืจื
For they will turn away your son from following me, that they may serve other gods; so will the anger of the Lord be kindled against you, and destroy you speedily.
7:5
ืื ืืึพืื ืชืขืฉื ืืื ืืืืืชืืื ืชืชืฆื ืืืฆืืชื ืชืฉืืจื ืืืฉืืจืื ืชืืืขืื ืืคืกืืืืื ืชืฉืจืคืื ืืืฉื
But thus shall you deal with them; you shall destroy their altars, and break down their images, and cut down their Asherim, and burn their carved idols with fire.
That's your choice. Is there any similarity between your worship and the practice of those seven unholy nations? According to the story, these seven commit "every abomination that the Lord hates."
Deuteronomy 12: ( emphasis mine )
12:29
ืืึพืืืจืืช ืืืื ืืืืื ืืชึพืืืืื ืืฉืจ ืืชื ืืึพืฉืื ืืจืฉืช ืืืชื ืืคื ืื ืืืจืฉืช ืืชื ืืืฉืืช ืืืจืฆืื
When the Lord your God shall cut off the nations from before you, where you go to dispossess them, and live in their land;
12:30
ืืฉืืจ ืื ืคืึพืชื ืงืฉ ืืืจืืื ืืืจื ืืฉืืื ืืคื ืื ืืคืึพืชืืจืฉ ืืืืืืื ืืืืจ ืืืื ืืขืืื ืืืืื ืืืื ืืชึพืืืืืื ืืืขืฉืึพืื ืืึพืื ืื
Take heed to yourself that you be not snared by following them, after they are destroyed from before you; and that you inquire not about their gods, saying, How did these nations serve their gods? that I may also do likewise.
12:31
ืืึพืชืขืฉื ืื ืืืืื ืืืืื ืื ืืึพืชืืขืืช ืืืื ืืฉืจ ืฉื ื ืขืฉื ืืืืืืื ืื ืื ืืชึพืื ืืื ืืืชึพืื ืชืืื ืืฉืจืคื ืืืฉ ืืืืืืืื
You shall not do so to the Lord your God; for every abomination to the Lord, which he hates, have they done to their gods; for even their sons and their daughters they have burned in the fire to their gods.