Exemestane With or Without Entinostat in Treating Patients With Recurrent Hormone Receptor-Positive Breast Cancer That is Locally Advanced or Metastatic

Inclusion Criteria: * Estrogen receptor (ER) and/or progesterone receptor (PR) positive histologically confirmed adenocarcinoma of the breast with staining of \>= 1% cells will be considered positive; receptor status may be based on any time during treatment prior to study randomization, and from any site (i.e. primary, recurrent, or metastatic) * Patients whose tumors have HER2 immunohistochemistry (IHC) 3+, in situ hybridization (ISH) \>= 2.0, or average HER2 copy number \>= 6.0 signals per cell are not eligible; receptor status may be based on any time during treatment prior to study randomization, and from any site (i.e. primary, recurrent, or metastatic) * Patients must have measurable or non-measurable stage III/locally advanced or metastatic carcinoma of the breast where local therapy with curative intent is not possible; lesions must be evaluated =\< 4 weeks prior to study randomization; diagnostic-quality computed tomography (CT) scans with both oral and intravenous (IV) contrast are the expected radiologic method, unless an alternative is approved * NOTE: Where baseline imaging has already been performed =\< 6 weeks prior to study randomization, repeat imaging may not be required * NOTE: As of October 16, 2016, accrual of new patients having non-measurable disease has stopped; the planned accrual for this target population has been reached * Pre/peri- and postmenopausal women and all men are eligible for this trial; postmenopausal is defined as: * Age \>= 55 years and one year or more of amenorrhea * Age \< 55 years and one year or more of amenorrhea, with estradiol \< 20 pg/ml * Age \< 55 with prior hysterectomy but intact ovaries, with estradiol \< 20 pg/ml * Prior bilateral oophorectomy * NOTE: Women who do not fit the criteria for being postmenopausal as above are deemed pre-or peri-menopausal; pre/perimenopausal women and all men can enroll provided they agree to receive concomitant luteinizing hormone-releasing hormone (LHRH) agonist; pre/perimenopausal women must have commenced treatment with LHRH agonist at least 4 weeks prior to randomization; if patients have received alternative LHRH agonist prior to study entry, they must switch to goserelin for the duration of the trial * Sexually active males and pre/perimenopausal women must agree to use an accepted and effective method of contraception or to abstain from sexual intercourse for the duration of their participation in the study and for 3 months after discontinuation of therapy * Women must not be pregnant or breast-feeding; all females of childbearing potential must have a blood test or urine study =\< 2 weeks prior to randomization * A female of childbearing potential is any woman, regardless of sexual orientation or whether they have undergone tubal ligation, who meets the following criteria: 1) has not undergone a hysterectomy or bilateral oophorectomy; or 2) has not been naturally postmenopausal for at least 24 consecutive months (i.e., has had menses at any time in the preceding 24 consecutive months) * Patients must not have known central nervous system metastasis or a history of central nervous system (CNS) metastases; patients with leptomeningeal disease are not eligible * Patients must be disease-free of prior invasive malignancies for \> 5 years with the exception of curatively-treated basal cell or squamous cell carcinoma of the skin or carcinoma in situ of the cervix * NOTE: If there is a history of prior malignancy, patients must not be receiving other specific treatment for that cancer * Patients must meet at least one of the following criteria: * Disease progression any time after non-steroidal AI use in the advanced disease setting * Relapse while on or within =\< 12 months of end of adjuvant non-steroidal AI therapy with or without prior endocrine therapy for advanced disease * NOTE: In either setting, treatment with any prior endocrine therapy must be completed \>= 2 weeks prior to cycle 1 day 1 (C1D1) of study treatment with the exception of exemestane which is permitted in the advanced disease setting within =\< 4 weeks immediately prior to C1D1; prior adjuvant exemestane is allowed if the disease free interval is \> 12 months from the discontinuation of exemestane; prior faslodex, everolimus, palbociclib or other cyclin-dependent kinase (CDK) inhibitor (e.g. ribociclib, abemaciclib) use are allowed and must have been completed \>= 2 weeks prior to C1D1; failure to adhere to this washout guideline will result in a protocol violation * Patients may have received only one prior chemotherapy regimen for metastatic disease provided treatment was completed \>= 3 weeks prior to randomization * Patients may be treated with bone modifying agents such as bisphosphonates or RANK-ligand agents (e.g. denosumab) per American Society of Clinical Oncology (ASCO) guidelines; whenever possible, patients requiring bone modifying agents should start treatment \>= 7 days prior to study therapy and should continue the same agent throughout study unless clinically compelled to change * Prior radiotherapy must in general have been completed \>= 2 weeks prior to randomization and patients must have recovered from the toxicity of the radiation * NOTE: Patients may receive concurrent radiation therapy to painful sites of bony disease or areas of impending fracture as long as sites of measurable or non-measurable disease outside the radiation therapy port are available to follow * Patients must NOT receive concurrent anti-cancer therapy or investigational agent unless specified in protocol * Patients must NOT be receiving valproic acid, an histone deacetylase (HDAC) inhibitor, and may not have previously received any HDAC inhibitor prior to enrollment (e.g. valproic acid, entinostat, vorinostat) unless discussed with the study chair; patients must not have received prior HDAC therapy for the treatment of their malignancy * Patients must have no known allergies to exemestane, entinostat, or medications that have a benzamide structure (e.g., tiapride, remoxipride, clebropride) * Patients must NOT suffer from medical or psychiatric conditions that would interfere with protocol compliance, the ability to provide informed consent, or assessment of response or anticipated toxicities; this includes uncontrolled intercurrent illness including, but not limited to ongoing or active infection * Patients must have recovered from all clinically relevant adverse events to grade 1 or baseline due to previous agents administered (except alopecia) * Patients must have adequate hematologic, liver and renal function =\< 28 days prior to randomization * NOTE: It is preferred that laboratory values for eligibility be assessed after the last dose of prior treatment, especially in cases where most-recent treatment prior to study entry is chemotherapy * Hemoglobin (HgB) \>= 9.0 g/dL (=\< 28 days prior to randomization) * Platelet count \>= 100,000/mcL (=\< 28 days prior to randomization) * Absolute neutrophil count \>= 1,500/mcL (=\< 28 days prior to randomization) * Creatinine =\< 2.0 mg/dL (=\< 28 days prior to randomization) * Total bilirubin \< 1.5 x institutional upper limit of normal (=\< 3 mg/dL in case of Gilbert's syndrome) (=\< 28 days prior to randomization) * Transaminases (alanine aminotransferase \[ALT\], aspartate aminotransferase \[AST\]) =\< 2.5 x institutional upper limit normal (=\< 28 days prior to randomization) * Known human immunodeficiency virus (HIV)-positive patients should have a cluster of differentiation (CD)4 count \> 250/mm\^3 * Patients must have Eastern Cooperative Oncology Group (ECOG) performance status 0-1 * Patients must have a life expectancy \>= 12 weeks * Patients must be able to swallow tablets