This is a phase I study which will test the safety of different doses of the patients own immune cells which have been changed to help recognize and destroy the cancer cells. The investigators want to find out what effects, good and/or bad, it has on the body and on the prostate cancer. The immune cells (T cells) used in this study will be the patients own immune cells. They will be removed from the patients blood, changed in the laboratory, and then put back into their body. T cells help the body fight infections. These cells may also kill cancer cells in some cases. Right now the patients T cells are unable to kill the cancer cells. For this reason, the physician will change the T cells by putting in a gene so that they may be able to better recognize and kill the prostate cancer cells. A gene is a portion of information which comes from the DNA and tells the cell what to do. This gene will be put into the patients T cells by a weakened virus. It is hoped that this approach will help the T cells recognize the prostate cancer tumor cells and possibly kill them. The investigators have found that T cells modified in this way were able to cure a cancer similar to Chronic Lymphocytic Leukemia in mice. However, this is an entirely new treatment for prostate cancer and it is not known if it will have any beneficial or unexpected harmful effects.

Inclusion Criteria:

• Male, over 18 years of age
• Karnofsky Performance Scale (KPS) greater or = to 70%
• Histologic confirmation of prostate cancer at Memorial Sloan-Kettering Cancer Center (MSKCC)
• A diagnosis of progressive castrate metastatic prostate cancer defined as one or more of the following three criteria:
• Soft tissue progression defined by RECIST 1.0
• Bone disease progression defined by PCWG2 with two or more new lesions on bone scan
• Post-hormonal therapy rising PSA values from a hormone therapy nadir on greater or = to 3 successive determinations at least two weeks apart, where the increase above the nadir is the greater of greater or = to 2.0 ng/mL or a greater or = to 10% change (Subjects with a rise in PSA but no evidence of metastases at any time by imaging studies will NOT be eligible.)
• For subjects who have discontinued anti-androgen therapy, PSA rise as defined above must be documented:
• Within two weeks after discontinuation if used following surgical or medical castration (second line therapy)
• After four weeks discontinuation if used as first line therapy.
• Evidence of metastatic disease in bone on bone scan, CT scan, and/or by MRI atany time following the initial diagnosis of prostate cancer.
• Castrate level of serum testosterone (<50 ng/mL) achieved by prior hormonal therapy consisting of either a) orchiectomy or b) luteinizing hormone-releasing hormone (LHRH) agonists with or without an anti-androgen
• Castrate level of serum testosterone (<50 ng/mL) achieved by prior hormonal therapy consisting of either a) orchiectomy or b) luteinizing hormone-releasing hormone (LHRH) agonists with or without an anti-androgen
• Lab requirements (Hematology):
• White blood count (WBC) ≥3,000/mm3
• Absolute neutrophil count ≥1,500/mm3
• Platelet ≥100,000/ mm3
• Hemoglobin ≥10 gm/d

Lab requirements (Serum Chemistry):

• Bilirubin <1.5 X ULN (the upper limit of normal) (Subjects with confirmed Gilbert's Disease as the cause of their elevated bilirubin are to be permitted.)
• Serum alanine aminotransferase/serum aspartate aminotransferase (ALT/AST) < 2.5 X ULN (the upper limit of normal)
• Serum creatinine <1.5 X ULN(the upper limit of normal)
• Negative screen for Human Immunodeficiency virus (HIV), Hepatitis B virus (HBV) antigen, and Hepatitis C virus (HCV). If testing was done within the past three months, there is no need to repeat testing, as long as documentation of results is provided to the study site. Subjects must receive counseling and sign a separate informed consent for HIV testing.
• Subjects and their partners of reproductive potential must agree to use an effective form of contraception during the period of drug administration and for four weeks following the completion of the last administration of the study drug. An effective form of contraception is defined as oral contraceptives plus one form of barrier method or double barrier methods (condom with spermicide or condom with diaphragm).
• Subjects must be able to understand the potential risks and benefits of the study, and be able to read and give written informed consent.

Exclusion Criteria:

• History of non-prostate, primary, malignant cancer, except for non-melanoma skin cancer within previous five years
• History of splenectomy
• Autoimmune- or Ab-mediated disease including, but not limited to, systemic lupus erythematosus, rheumatoid arthritis, ulcerative colitis, Crohn's disease, temporal arteritis, and thyroiditis
• Clinically significant cardiac disease (New York Heart Association Class III/IV) or severe debilitating pulmonary disease
• Radiation therapy within four weeks prior to start of study treatment (Day -1)
• Patients may not have received more than one prior chemotherapy
• The following medications within four weeks prior to start of study treatment (Week 1): systemically administered radiopharmaceuticals such as bone seeking isotopes (e.g., samarium-153 Lexidronam); hematopoietic growth factors other than erythropoietin; medroxyprogesterone as an appetite stimulant; or alternative medicine treatments for prostate cancer, including Prostasol (formerly: PC-Plus), saw palmetto, or Zyflamend®
• Active central nervous system (CNS) or symptomatic epidural metastatic disease
• An infection requiring antibiotic treatment within seven days of starting study treatment (Day -1)
• A requirement for daily systemic corticosteroids for any reason; or other immunosuppressive or immunomodulatory agents. Topical, nasal or physiologic corticosteroids are to be permitted.
• Administration of live attenuated vaccines within eight weeks of start of study treatment (Day -1) and throughout the study
• Administration of subunit or killed vaccines, such as influenza or pneumococcal vaccine, within two weeks prior to study treatment (Day-1) and throughout the study; EXCEPTION - Vaccination for influenza is permitted between Week 12 through Week 16 and after Week 20.
• Positive stool guaiac, excluding hemorrhoids or documented radiation-induced proctitis if test is performed at the discretion of the treating physician (stool guaiac test is not required to screen for eligibility).
• Any other medical condition that in the opinion of the Investigator may interfere with a subject's participation in, or compliance with, the study
• Participation in a therapeutic research study or receipt of an investigational drug within 4 weeks leukapheresis.
• Allergy to ganciclovir or acyclovir.