This is an open-label, dose-escalation Phase 1/2 study to assess the safety of ASTX660, determine the maximum tolerated dose (MTD), recommended Phase 2 dose (RP2D), and recommended dosing regimen, and to obtain preliminary efficacy, pharmacokinetic (PK), and target engagement data, in subjects with advanced solid tumors or lymphoma for whom standard life-prolonging measures are not available.

ASTX660 is a synthetic small molecule dual antagonist of cellular inhibitor of apoptosis protein (cIAP) 1 and X-linked inhibitor of apoptosis protein (XIAP) that has been shown to have potent proapoptotic and tumor growth inhibitory activity in nonclinical models. The Phase 1 portion of the study (completed) will determine the MTD, RP2D, and recommended dosing regimen. The Phase 2 portion will evaluate activity in selected tumor types. Subjects will continue to receive their assigned treatment throughout the study until the occurrence of disease progression, death, or unacceptable treatment-related toxicity, or until the study is closed by the sponsor.

Inclusion Criteria:

1. Able to understand and comply with the protocol and study procedures, understand the risks involved in the study, and provide written informed consent before any study-specific procedure is performed.

2. Men and women 18 years of age or older.

3. Participants with histologically or cytologically confirmed advanced solid tumors or lymphoma that is metastatic or unresectable, and for whom standard life-prolonging measures are not available. Specific tumor types that will be selected for study in Phase 2 are detailed in the protocol.

   a. For Phase 2 Cohort 3, participants must have histologically confirmed PTCL (local pathology report) as defined by 2016 World Health Organization (WHO) classification. The following subtypes are eligible for the study: adult T-cell lymphoma/leukemia, extranodal natural killer (NK)/T-cell lymphoma nasal type, enteropathy-associated T-cell lymphoma, monomorphic epitheliotropic intestinal T-cell lymphoma, hepatosplenic T-cell lymphoma, subcutaneous panniculitis-like T-cell lymphoma, peripheral T-cell lymphoma not otherwise specified, angioimmunoblastic T-cell lymphoma, follicular T-cell lymphoma, nodal peripheral T-cell with T-follicular helper (THF) phenotype, and anaplastic large-cell lymphoma.

4. For Phase 2 Cohorts 3 and 4, participants must have evidence of documented progressive disease and must have received at least two prior systemic therapies.

   1. Participants with CD30-positive lymphoma must have received, be ineligible for, or intolerant to brentuximab vedotin, provided that brentuximab vedotin is locally approved and available.
   2. Participants with mycosis fungoides or Sezary syndrome must have received, be ineligible or intolerant to mogamulizumab, provided that mogamulizumab is locally approved and available.

5. In the Phase 2 portion of the protocol only, participants must have measurable disease according to response criteria appropriate for their type of cancer.

   a. For Phase 2 Cohort 3 (PTCL), measurable disease by contrast-enhanced diagnostic CT (at least 1 nodal lesion >1.5 cm or extranodal lesions >1.0 cm) is required.

6. Eastern Cooperative Oncology Group (ECOG) performance status of 0, 1, or 2.

7. Acceptable organ function, as evidenced by the following laboratory data:

   1. Aspartate aminotransferase (AST) and alanine aminotransferase (ALT) <=2.0 * upper limit of normal (ULN).

   2. Total serum bilirubin <=1.5 * ULN

   3. Absolute neutrophil count (ANC):

      • Phase 1 and 2 (except Phase 2 participants with known lymphoma; ie, not applicable for Cohorts 3 or 4) >=1500 cells/mm3
      • Phase 2 participants with known lymphoma: >=1000 cells/mm3 (>750 cell/mm3 for participants with lymphoma in bone marrow)

   4. Platelet count:

      • Phase 1 and 2 (except Phase 2 participant with known lymphoma; ie, not applicable for Cohorts 3 or 4) >=100,000 cells/mm3
      • Phase 2 participants with known lymphoma: >= 50,000 cells/mm3; >=25,000 cells/mm3 for participants with lymphoma in bone marrow

   5. Serum creatinine levels <= 1.5 * ULN, or calculated (by Cockcroft-Gault formula or other accepted formula) or measure creatinine clearance >=50 mL/min.

   6. Amylase and lipase <=ULN.

8. Women of child-bearing potential (according to recommendations of the Clinical Trial Facilitation Group [CTFG]; see protocol for details) must not be pregnant or breastfeeding and must have a negative pregnancy test at screening. Women of child-bearing potential and men with female partners of child-bearing potential must agree to practice 2 highly effective contraceptive measures of birth control (as described in the protocol) and must agree not to become pregnant or father a child while receiving treatment with study drug and for at least 3 months after completing treatment. Contraceptive measures which may be considered highly effective comprise combined hormonal contraception (oral, vaginal, or transdermal) or progestogen-only hormonal contraception (oral, injectable, implantable) associated with inhibition of ovulation, intrauterine device, intrauterine hormone-releasing system, bilateral tubal occlusion, sexual abstinence, and surgically successful vasectomy. Abstinence is acceptable only if it is consistent with the preferred and usual lifestyle of the participant. Periodic abstinence (eg, calendar, ovulation, symptothermal, or postovulation methods) and withdrawal are not acceptable methods of birth control.

Exclusion Criteria:

1. Hypersensitivity to ASTX660, excipients of the drug product, or other components of the study treatment regimen.

2. Poor medical risk because of systemic diseases (e.g. active uncontrolled infections) in addition to the qualifying disease under study.

3. Life-threatening illness, significant organ system dysfunction, or other condition that, in the investigator's opinion, could compromise participant safety or the integrity of the study outcomes, or interfere with the absorption or metabolism of ASTX660.

4. History of, or at risk for, cardiac disease, as evidenced by 1 or more of the following conditions:

   1. Abnormal left ventricular ejection fraction (LVEF; <50%) or echocardiogram ECHO or multiple gated acquisition scan (MUGA).
   2. Congestive cardiac failure of >= Grade 3 severity according to New York Heart Association (NYHA) functional classification defined as participants with marked limitation of activity and who are comfortable only at rest.
   3. Unstable cardiac disease including angina or hypertension as defined by the need for overnight hospital admission within the last 3 months (90 days).
   4. History or presence of complete left bundle branch block, heart block, cardiac pacemaker or significant arrhythmia.
   5. Concurrent treatment with any medical that prolongs QT interval and may induce torsades de pointes, and which cannot be discontinued at least 2 weeks before treatment with ASTX660. [Applies to Phase 1 only].
   6. Personal history of long QTc syndrome or ventricular arrhythmias including ventricular bigeminy.
   7. Screening 12-lead ECG with measurable QTc interval (according to either Fridericia's or Bazett's correction) of >=470 msec).
   8. Any other condition that, in the opinion of the investigator, could put the participant at increased cardiac risk.

5. Known history of human immunodeficiency virus (HIV) infection, or seropositive results consistent with active hepatitis B virus (HBV) or active hepatitis C virus (HCV) infection.

6. Grade 2 or greater neuropathy [Applies to Phase 1]. Grade 3 or greater neuropathy [Applies to Phase 2].

7. Known brain metastases, unless stable or previously treated.

8. Known significant mental illness or other conditions such as active alcohol or other substance abuse that, in the opinion of the investigator, predisposes the participant to high risk of noncompliance with the protocol treatment or assessments.

9. Prior anticancer treatments or therapies within the indicated time window prior to first dose of study treatment (ASTX660), as follows:

   1. Cytotoxic chemotherapy or radiotherapy within 3 weeks prior and any encountered treatment-related toxicities (excepting alopecia) not resolved to Grade 1 or less [Phase 1] or Grade 2 or less [Phase 2].
   2. Skin directed treatments, including topicals and radiation within 2 weeks prior.
   3. Monoclonal antibodies within 4 weeks prior and any encountered treatment-related toxicities not resolved to Grade 1 or less [Phase 1] or Grade 2 or less [Phase 2].
   4. Small molecules or biologics (investigational or approved) within the longer of 2 weeks or 5 half-lives prior to study treatment and any encountered treatment-related toxicities not resolved to Grade 1 or less [Phase 1] or Grade 2 or less [Phase 2].
   5. At least 6 weeks must have elapsed since CAR-T infusion and participants must have experienced disease progression, and not have residual circulating CAR-T cells in peripheral blood (based on local assessment). Any encountered treatment-related toxicities must have resolved to Grade ≤1.

10. Concurrent second malignancy currently requiring active therapy, except breast or prostate cancer stable on or responding to endocrine therapy or superficial bladder cancer [Phase 2].

11. Known central nervous system (CNS) lymphoma [Phase 2].

12. Participants with a history of allogenic transplant must not have ≥Grade 3 graft-versus-host disease (GVHD) or any clinically significant GVHD requiring systemic immunosuppression [Phase 2].

13. Systemic corticosteroids >20 mg prednisone equivalent (unless participant has been taking a continuous dose for >3 weeks prior to study entry and there is documented radiological progression) [Phase 2]. Stable dose of medium or low potency topical corticosteroids for at least 3 weeks prior to study entry are permitted [Phase 2].