Move file

Move-Item C:\MyDirectory1\MyFile.txt C:\MyDirectory2\ -force

-force causes an overwrite if necessary.

You can provide a full target path to perform a rename at the same time if wanted.