diff --git a/data/auto_parse/level_freeze/frozen/idx_15.jsonl b/data/auto_parse/level_freeze/frozen/idx_15.jsonl new file mode 100644 index 0000000..4b55ed4 --- /dev/null +++ b/data/auto_parse/level_freeze/frozen/idx_15.jsonl @@ -0,0 +1,46 @@ +{"idx": 15, "order": 0, "level": 0, "span": "SEPARATION AGREEMENT\nAND\nGENERAL RELEASE OF CLAIMS"} +{"idx": 15, "order": 1, "level": 1, "span": "THIS SEPARATION AGREEMENT AND GENERAL RELEASE OF CLAIMS (hereinafter “Agreement”) is entered into by and between Brian M. Culley (hereinafter “Employee”) and Mast Therapeutics, Inc. (hereinafter “Mast” or the “Company”).  Employee and Mast hereinafter are collectively referred to as the “Parties” or individually referred to as a “Party.”"} +{"idx": 15, "order": 2, "level": 1, "span": "RECITALS"} +{"idx": 15, "order": 3, "level": 1, "span": "A.Mast is a corporation and is doing business in the State of California."} +{"idx": 15, "order": 4, "level": 1, "span": "B.Employee’s employment with Mast as a Chief Executive Officer is expected to terminate as of the closing (the “Closing”) of the acquisition of Savara Inc. (the “Change of Control”), which is expected to occur on or about April 21, 2017 (such date of termination of employment the “Termination Date”)."} +{"idx": 15, "order": 5, "level": 1, "span": "C.In accordance with the terms of the Executive Severance Agreement, dated March 23, 2016, between Employee and Mast (the “Executive Severance Agreement”), Employee desires to settle and compromise any and all possible claims and disputes he/she has against any of the Releasees, as defined below, arising out of their relationship to date, and to provide for a general release of any and all such claims."} +{"idx": 15, "order": 6, "level": 1, "span": "AGREEMENT"} +{"idx": 15, "order": 7, "level": 1, "span": "1.Termination of Employment and Resignation of Positions\nEmployee agrees that his/her employment with Mast will terminate as part of the Closing effective as of the Termination Date and he/she has complied, or will comply as of the Termination Date, as applicable, with the provisions of Section 1.3 of the Executive Severance Agreement.  Employee hereby resigns, effective as of the Termination Date, any and all other positions he/she holds with Mast and any of its subsidiaries, including positions as a director of Mast or any of its subsidiaries.  In the event that Employee’s employment with Mast is not terminated in connection with the Closing, this Agreement shall automatically terminate and no longer remain in force or effect without further obligation of either of the Parties."} +{"idx": 15, "order": 8, "level": 1, "span": "2.Separation Pay/Consideration\nIn consideration of the covenants and releases given herein, upon termination of Employee’s employment on the Termination Date, and subject to non-revocation of this Agreement as set forth in Section 4.c. and execution of the Affirmation (as defined below), Employee will become eligible to receive the following consideration:\na.Separation Pay.  Mast will tender a check to Employee in an amount of Nine Hundred Twenty-Three Thousand, Five Hundred Eighteen Dollars and Eighty Cents ($923,518.80), less applicable federal and California payroll tax deductions, which is the equivalent of (i) twenty-four (24) months of Employee’s base salary and (ii) the amount equal to the premiums necessary to continue Employee’s health insurance coverage in effect for Employee and Employee’s covered dependents under the Consolidated Omnibus Reconciliation Act of 1985, for a period of twenty-four (24) months; and\nb.Unemployment Insurance Claim.  Mast will not oppose Employee’s claim for unemployment insurance benefits, and, if asked, will inform the California Employment Development Department that Employee was laid off by Mast as part of the Change in Control."} +{"idx": 15, "order": 9, "level": 1, "span": "3.Release\na.Release.  Employee does hereby unconditionally, irrevocably and absolutely release and discharge Mast and its current and former officers, directors, employees, agents, investors, attorneys, shareholders, administrators, affiliates, benefit plans, plan administrators, professional employer organization or co-employer,\ninsurers, trustees, divisions, and subsidiaries, and predecessor and successor corporations and assigns, (collectively, the “Releasees”) from any and all loss, liability, claims, demands, causes of action or suits of any type, whether in law and/or in equity, related directly or indirectly, or in any way connected with any transactions, affairs or occurrences between them to date, including, but not limited to, Employee’s employment with Mast and the termination of said employment.  Employee agrees that the foregoing consideration represents settlement in full of all outstanding obligations owed to Employee by the Releasees, other than consideration to which Employee may be entitled in respect of (i) a Change of Control, and (ii) unpaid wages, accrued and unused vacation and reimbursement for business expenses validly incurred prior to termination.  This Agreement specifically applies, without limitation, to any and all disputed wage claims, claims for unpaid expenses, contract claims, tort claims, claims for wrongful termination, and claims arising under Title VII of the Civil Rights Act of 1991, the Americans with Disabilities Act, the Equal Pay Act, the Worker Adjustment and Retraining Notification Act, the Employee Retirement Income Security Act, the Sarbanes-Oxley Act of 2002, the California Fair Employment and Housing Act, the Fair Labor Standards Act, the Family and Medical Leave Act, the California Family Rights Act, the California Labor Code, the California Business and Professions Code, and any and all federal or state statutes or laws governing employment and/or discrimination in employment.  In addition, this Agreement specifically applies to any claims for age discrimination harassment or retaliation in employment, including any claims arising under the Age Discrimination in Employment Act or any other statutes or laws which govern age discrimination in employment.  Nothing in this Agreement shall be construed to mean that Employee is releasing or waiving claims to enforce this Agreement, workers’ compensation claims, claims for unemployment insurance benefits, claims for any vested retirement, any claim for indemnification (including under the Company’s organizational documents or insurance policies) arising in connection with an action instituted by a third party against the Company or Employee, or claims that, by law, cannot be waived.\nb.Section 1542 Waiver.  Employee does expressly waive all of the benefits and rights granted to him/her pursuant to California Civil Code section 1542, which reads as follows:\nA general release does not extend to claims which the creditor does not know of or suspect to exist in his or her favor at the time of executing the release, which if known by him or her must have materially affected his or her settlement with the debtor.\nEmployee does certify that he/she has read all of this Agreement, including the release provisions contained herein and the quoted Civil Code section, and that he/she fully understands all of the same.  Employee hereby expressly agrees that this Agreement shall extend and apply to all unknown, unsuspected and unanticipated injuries and damages, as well as those that are now disclosed.\nc.No Further Action.  Except as set forth in Section 5, Employee irrevocably and absolutely agrees that he/she will not prosecute nor allow to be prosecuted on his/her behalf, in any administrative agency, whether federal or state, or in any court, whether federal or state, any claim or demand of any type related to the matters released above, it being the intention of the Parties that with the execution by Employee of this release, the Releasees will be absolutely, unconditionally and forever discharged of and from all obligations to or on behalf of Employee related in any way to the matters discharged herein."} +{"idx": 15, "order": 10, "level": 1, "span": "4.Additional Provisions Regarding Release of Age Claims/OWBPA Provisions\na.ADEA Claims.  This section of the Agreement exclusively addresses issues relating to Employee’s release of claims arising under federal law involving discrimination on the basis of age in employment (age forty and above).  This section is provided separately, in compliance with federal law, including but not limited to the Older Workers’ Benefit Protection Act of 1990 (“OWBPA”), to ensure that Employee clearly understands his rights so that any release of age discrimination claims under federal law (the ADEA) is knowing and voluntary on the part of Employee.\nb.Review Period/OWBPA Provisions.  In accordance with the provisions of the OWBPA, Employee is aware of the following:  Employee represents, acknowledges and agrees that Mast has advised him/her, in writing, (i) to discuss this Agreement with an attorney and to that extent, if any, that Employee has desired, Employee has done so; (ii) that Mast has given Employee forty-five (45) days from receipt of this Agreement to review and consider this Agreement before signing it, and Employee understands that he/she may use as much of this forty-"} +{"idx": 15, "order": 11, "level": 1, "span": "- 2 -\nfive (45) day period as he/she wishes prior to signing; (iii) that no promise, representation, warranty or agreements not contained herein have been made by or with anyone to cause him/her to sign this Agreement; (iv) that he/she has read this Agreement in its entirety, and fully understands and is aware of its meaning, intent, content and legal effect; (v) that he/she is executing this release voluntarily and free of any duress or coercion; (vi) that this Agreement includes rights and claims under the federal Age Discrimination in Employment Act, as amended, and the federal OWBPA, as amended; and (vii) that this Agreement does not waive rights or claims that may arise after the date Employee signs this Agreement.\nc.Effective Date of Agreement.  The Parties acknowledge that for a period of seven (7) days following the execution of this Agreement, Employee may revoke the Agreement, and the Agreement shall not become effective or enforceable until the revocation period has expired.  This Agreement shall become effective eight (8) days after it has been signed by Employee and Mast, and in the event the parties do not sign on the same date, then this Agreement shall become effective eight (8) days after the date it is signed by Employee."} +{"idx": 15, "order": 12, "level": 1, "span": "5.Protected Rights\nEmployee understands that nothing contained in this Agreement limits Employee’s ability to file a charge or complaint with the Equal Employment Opportunity Commission, the National Labor Relations Board, the Occupational Safety and Health Administration, the Securities and Exchange Commission, or any other federal, state or local government agency or commission (“Government Agencies”), including an Age Discrimination in Employment Act charge or complaint, although Employee may have no right to relief by reason of the claims Employee has released herein.  Employee further understands that this Agreement does not limit Employee’s ability to communicate with any Government Agencies or otherwise participate in any investigation or proceeding that may be conducted by any Government Agency, including providing documents or other information, without notice to Mast.  Nothing in this Agreement shall restrict or limit any right Employee may have to receive a whistleblower award or bounty for information provided to the Securities and Exchange Commission.  In addition, pursuant to the Defend Trade Secrets Act of 2016, Employee is notified that an individual will not be held criminally or civilly liable under any federal or state trade secret law for the disclosure of a trade secret that (i) is made in confidence to a federal, state, or local government official (directly or indirectly) or to an attorney solely for the purpose of reporting or investigating a suspected violation of law, or (ii) is made in a complaint or other document filed in a lawsuit or other proceeding, if (and only if) such filing is made under seal. In addition, an individual who files a lawsuit for retaliation by an employer for reporting a suspected violation of law may disclose the trade secret to the individual’s attorney and use the trade secret information in the court proceeding, if the individual files any document containing the trade secret under seal and does not disclose the trade secret, except pursuant to court order."} +{"idx": 15, "order": 13, "level": 1, "span": "6.No Cooperation\nSubject to Section 5 governing Employee’s Protected Rights, Employee agrees that he/she will not knowingly encourage, counsel, or assist any attorneys or their clients in the presentation or prosecution of any disputes, differences, grievances, claims, charges, or complaints by any third party against the Company and its subsidiaries, and predecessor and successor corporations and assigns, unless under a subpoena or other court order to do so or as related directly to the ADEA waiver in this Agreement. Employee agrees both to promptly notify the Company and its successor corporations, upon receipt of any such subpoena or court order, and to furnish, within three (3) business days of its receipt, a copy of such subpoena or other court order. If approached by anyone for counsel or assistance in the presentation or prosecution of any disputes, differences, grievances, claims, charges, or complaints against the Company and its subsidiaries, and predecessor and successor corporations and assigns, Employee shall state no more than that Employee cannot provide counsel or assistance."} +{"idx": 15, "order": 14, "level": 1, "span": "7.Acknowledgements/Affirmations\nEmployee acknowledges and affirms that he/she has been paid and/or has received all wages, bonuses, incentive compensation, accrued vacation and benefits to which Employee may be entitled, except for such wages, bonuses, incentive compensation, accrued vacation and benefits to which Employee may be entitled as a result of the Change of Control and/or termination of employment.  Employee also acknowledges and affirms that he/she has been provided information regarding his/her inability to continue to receive health insurance benefits as COBRA benefits after the termination of his/her employment due to Mast’s termination, in connection with the Change in Control, of the health insurance plans in which Employee has participated.  Employee further acknowledges and affirms that he/she has returned all documents and other items provided to Employee by the Company (with the exception of a copy of the Employee Handbook and personnel documents specifically relating to Employee), developed or obtained by Employee in connection with Employee’s employment with the Company, or otherwise belonging to the Company."} +{"idx": 15, "order": 15, "level": 1, "span": "- 3 -"} +{"idx": 15, "order": 16, "level": 1, "span": "8.Confidentiality/Non-Disparagement\nEmployee agrees that all matters relative to this Agreement shall remain confidential.  Accordingly, Employee hereby agrees that, with the exception of his/her spouse, counsel and tax advisors, he/she shall not discuss, disclose or reveal to any other persons, entities or organizations, whether within or outside of Mast, the terms and conditions of this Agreement.  The Parties acknowledge, however, that Mast may be required to file a copy of this Agreement with the Securities and Exchange Commission, in which case, the terms and conditions of this Agreement will be accessible for review by the public.  Nothing in this section prevents Employee from disclosing to any third party that his/her employment with Mast terminated in connection with the Change in Control.  Employee agrees not to make any derogatory or adverse statements, written or verbal, regarding the Releasees to anyone, and agrees to refrain from knowingly interfering in any tortious manner with the contracts and relationships of the Company.  Mast agrees not to make any derogatory or adverse statements, written or verbal, regarding Employee to anyone.  Employee understands that the Company’s obligations under this paragraph extend only to the Company’s current executive officers and members of its Board of Directors and only for so long as each officer or member is an employee or Director of the Company."} +{"idx": 15, "order": 17, "level": 1, "span": "9.Affirmation of Release and Waiver\nPrior to receipt of the consideration set forth in Section 2, Employee shall execute and deliver the Affirmation in substantially the form set forth in Exhibit A (the “Affirmation”)."} +{"idx": 15, "order": 18, "level": 1, "span": "10.Reference Requests\nAny reference requests concerning Employee will be referred to the Human Resources Department.  The only information that will be provided in response to such a request will be Employee’s dates of employment, his/her title, confirmation of his/her rate of pay, a statement that Employee was terminated in connection with the Change in Control and would not have been terminated but for that company action, and a statement that it is Mast’s policy to only provide that information."} +{"idx": 15, "order": 19, "level": 1, "span": "11.Tax Consequences\nThe Company makes no representations or warranties with respect to the tax consequences of the payments and any other consideration provided to Employee or made on Employee’s behalf under the terms of this Agreement.  Employee agrees and understands that Employee is responsible for payment, if any, of local, state, and/or federal taxes on the payments and any other consideration provided hereunder by the Company and any penalties or assessments thereon."} +{"idx": 15, "order": 20, "level": 1, "span": "12.Entire Agreement\nThe Parties further declare and represent that no promise, inducement or agreement not herein expressed has been made to them and that this Agreement together with the Executive Severance Agreement contain the full and entire agreement between and among the Parties, and that the terms of this Agreement are contractual and not a mere recital."} +{"idx": 15, "order": 21, "level": 1, "span": "13.Applicable Law\nThe validity, interpretation, and performance of this Agreement shall be construed and interpreted according to the laws of the State of California."} +{"idx": 15, "order": 22, "level": 1, "span": "14.Dispute Resolution\nExcept as set forth in Section 5, any dispute arising out of or related to this Agreement shall be resolved through binding arbitration through JAMS in San Diego, California, under the then current applicable rules of JAMS.  The arbitrator may grant injunctions and other relief in such disputes.  The arbitrator shall administer and conduct any arbitration in accordance with California law, including the California Code of Civil Procedure, and the arbitrator shall apply substantive and procedural California law to any dispute or claim, without reference to any conflict-of-law provisions of any jurisdiction.  To the extent that the JAMS rules conflict with California law, California law shall take precedence.  The decision of the arbitrator shall be final, conclusive, and binding on the parties to the arbitration.  Each party shall be responsible for its or his or her own costs and attorneys’ fees in connection with the arbitration, as well as half of the costs of the arbitration.  THE PARTIES HEREBY AGREE TO WAIVE THEIR RIGHT TO HAVE ANY DISPUTE BETWEEN THEM RESOLVED IN A COURT OF LAW BY A JUDGE OR JURY.  Notwithstanding the foregoing, this Section will not prevent either Party from seeking injunctive relief (or any other provisional remedy) from any court having jurisdiction over the Parties and the subject matter of their dispute relating to this Agreement."} +{"idx": 15, "order": 23, "level": 1, "span": "15.Knowing and Voluntary Agreement\nEmployee acknowledges that he/she has carefully read and fully understands all the provisions and effects of this Agreement.  Employee further acknowledges that he/she has been given the opportunity to consult with his/her own independent legal counsel and tax professional with respect to the matters referenced in this Agreement.  Employee acknowledges that he/she has fully discussed this Agreement with his/her attorney or has voluntarily chosen to sign this Agreement without consulting an attorney and/or tax"} +{"idx": 15, "order": 24, "level": 1, "span": "- 4 -\nprofessional, fully understanding the consequences of this Agreement.  Employee further acknowledges that he/she is entering into this Agreement without coercion or duress from any of the Releasees and that none of the Releasees have made any representations or promises concerning the terms or effects of this Agreement other than those set forth in this Agreement."} +{"idx": 15, "order": 25, "level": 1, "span": "16.Complete Defense\nThis Agreement may be pleaded as a full and complete defense and may be used as the basis for an injunction against any action, suit or proceeding which may be prosecuted, instituted or attempted by either party in breach thereof."} +{"idx": 15, "order": 26, "level": 1, "span": "17.Counterparts\nThis Agreement may be executed in counterparts and, if so executed, each such counterpart shall have the force and effect of an original.  A facsimile signature shall have the same force and effect as an original signature."} +{"idx": 15, "order": 27, "level": 1, "span": "18.Severability\nIf any provision of this Agreement, or part thereof, is held invalid, void or voidable as against public policy or otherwise, the invalidity shall not affect other provisions, or parts thereof, which may be given effect without the invalid provision or part.  To this extent, the provisions, and parts thereof, of this Agreement are declared to be severable."} +{"idx": 15, "order": 28, "level": 1, "span": "19.No Admission of Liability\nIt is understood that this Agreement is not an admission of any liability by any person, firm, association or corporation but is in compromise of a disputed claim."} +{"idx": 15, "order": 29, "level": 1, "span": "20.Successors and Assigns\nThis Agreement shall be binding upon and inure to the benefit of the Parties and their respective heirs, legal representatives, successors and assigns."} +{"idx": 15, "order": 30, "level": 1, "span": "IN WITNESS WHEREOF, the undersigned have executed this Agreement on the dates shown below."} +{"idx": 15, "order": 31, "level": 2, "span": "/s/ Brian M. Culley\nBrian M. Culley"} +{"idx": 15, "order": 32, "level": 2, "span": "Dated:"} +{"idx": 15, "order": 33, "level": 2, "span": "4-10-17"} +{"idx": 15, "order": 34, "level": 2, "span": "/s/ Brian M. Culley"} +{"idx": 15, "order": 35, "level": 2, "span": "Brian M. Culley"} +{"idx": 15, "order": 36, "level": 2, "span": "Mast Therapeutics, Inc."} +{"idx": 15, "order": 37, "level": 2, "span": "April 13, 2017"} +{"idx": 15, "order": 38, "level": 2, "span": "By:"} +{"idx": 15, "order": 39, "level": 2, "span": "/s/ Brandi L. Roberts"} +{"idx": 15, "order": 40, "level": 2, "span": "Name:"} +{"idx": 15, "order": 41, "level": 2, "span": "Brandi L. Roberts"} +{"idx": 15, "order": 42, "level": 2, "span": "Title:"} +{"idx": 15, "order": 43, "level": 2, "span": "Chief Financial Officer"} +{"idx": 15, "order": 44, "level": 1, "span": "- 5 -"} +{"idx": 15, "order": 45, "level": 1, "span": "Affirmation\nThe undersigned hereby acknowledges his/her termination of employment with the Company as of April 27, 2017 (the “Termination Date”) and further affirms that terms of the Separation Agreement and General Release of Claims between the undersigned and Mast Therapeutics, Inc. (the “Agreement”) remain in full force and effect as of the Termination Date, including, but not limited to, the release, waivers and affirmations set forth in Sections 3, 4 and 6 of the Agreement.\nThe undersigned acknowledges and affirms that he/she has been paid and/or has received all wages, bonuses, incentive compensation, accrued vacation and benefits to which the undersigned may be entitled, other than shares of the Company’s common stock pursuant to settlement of the restricted stock unit award granted to the undersigned in January 2017 pursuant to the Notice of Grant of Restricted Stock Units and Restricted Stock Units Award Agreement between the undersigned and the Company (the “RSUs Agreement”).  Upon receipt of 33,831 shares of the Company’s common stock (which is the amount granted under the RSUs Agreement as adjusted for the 70-for-1 reverse stock split implemented by the Company on April 27, 2017) in the undersigned’s E*Trade account, the undersigned acknowledges and affirms that he/she will have received all shares of Company common stock due to the undersigned pursuant to the RSUs Agreement.\nDated: April 27, 2017"} diff --git a/data/auto_parse/level_freeze/state.json b/data/auto_parse/level_freeze/state.json index cd17fb2..b0362b1 100644 --- a/data/auto_parse/level_freeze/state.json +++ b/data/auto_parse/level_freeze/state.json @@ -15,7 +15,8 @@ 11, 12, 13, - 14 + 14, + 15 ], "history": [ { @@ -226,6 +227,24 @@ "action": "freeze", "idx": 14, "n_records": 381 + }, + { + "ts": "2026-05-17T09:45:14", + "action": "freeze", + "idx": 15, + "n_records": 41 + }, + { + "ts": "2026-05-17T09:57:58", + "action": "freeze", + "idx": 15, + "n_records": 46 + }, + { + "ts": "2026-05-17T10:01:26", + "action": "freeze", + "idx": 15, + "n_records": 46 } ] } diff --git a/scripts/parse_doc2dict_with_config.py b/scripts/parse_doc2dict_with_config.py index 6e09ee7..4245e14 100644 --- a/scripts/parse_doc2dict_with_config.py +++ b/scripts/parse_doc2dict_with_config.py @@ -2400,6 +2400,165 @@ def _looks_like_title_continuation(title: str) -> bool: return True +# Bare single-word AGREEMENT/PLAN title — matches "AGREEMENT", "PLAN", +# "Agreement", "Plan" alone with nothing else. Used by +# `_swap_synthetic_l0_with_real_title` to detect doc2dict synthetic L0s +# (a body-internal section separator typeset as a centered bold paragraph +# that doc2dict's HTML walker promoted to L0 because it parents the body +# clauses). The REAL title carrier sits earlier in document order with a +# descriptive prefix ("SEPARATION AGREEMENT", "EMPLOYMENT AGREEMENT") and +# was demoted to L1 by `walk_sections`' no-children guard. +_BARE_AGREEMENT_PLAN_TITLE_RE = re.compile( + r"^\s*(?:AGREEMENT|PLAN)\s*$", + re.IGNORECASE, +) + + +def _swap_synthetic_l0_with_real_title( + rows: list[dict[str, Any]], +) -> list[dict[str, Any]]: + """Swap a synthetic single-word L0 with the real title carrier earlier + in document order. + + Some EX-10 filings typeset the agreement title as a centered bold + paragraph at the top (e.g. "SEPARATION AGREEMENT
AND
GENERAL + RELEASE OF CLAIMS"). Later, a body-internal section separator — + another centered bold paragraph containing just the word "AGREEMENT" + — sits between the RECITALS and the numbered sections. doc2dict's + HTML walker promotes that separator to be the parent of every + numbered body clause, and `walk_sections` then assigns it depth=0 + because (a) its title "AGREEMENT" matches the AGREEMENT|PLAN root + pattern and (b) it has direct children. + + Meanwhile, the real title-line records ("SEPARATION AGREEMENT" / + "AND" / "GENERAL RELEASE OF CLAIMS") sit earlier as siblings with no + children — they get demoted to depth=1 by the no-children guard in + `walk_sections`. The third record carries the preamble paragraph as + body_direct. + + The structural fingerprint of this pathology: + 1. The current L0 title is BARE: matches exactly "AGREEMENT" or + "PLAN" with nothing else. + 2. The current L0 has NO body_direct. + 3. There is an EARLIER record (smaller node_id) whose title matches + the AGREEMENT|PLAN end pattern AND has descriptive prefix words + (i.e. title is longer than just the bare word). + 4. That earlier record is in agreement scope (not envelope, not + trailer). + + Fix: swap depths. The earlier substantive-title record becomes + depth=0 (the real L0); the synthetic separator becomes depth=1 (a + regular body record carrying the source word "AGREEMENT" / "PLAN"). + + Runs BEFORE `_merge_multiline_l0_title` so the continuation walk + finds the correct L0 to expand. Runs BEFORE + `_split_l0_title_from_preamble` so the preamble body on the new L0 + gets split out into its own L1 record. + + No phrase matching — the detection is purely structural (depth, + body presence, title shape). + """ + if not rows: + return rows + + # Find the current L0. + l0: dict[str, Any] | None = None + for r in rows: + if ( + r.get("depth") == 0 + and not r.get("is_envelope") + and r.get("scope") == "agreement" + ): + l0 = r + break + if l0 is None: + return rows + + # Step 1: L0 title must be a BARE single-word AGREEMENT or PLAN. + # That's the structural fingerprint of doc2dict's synthetic section + # separator promoted to L0. Real agreement titles are longer than + # just the bare word ("CREDIT AGREEMENT", "INDEMNIFICATION + # AGREEMENT", etc.). A bare "AGREEMENT"/"PLAN" L0 always indicates + # that the actual title is elsewhere (typically earlier in the + # source as a multi-line centered-bold paragraph). + l0_title = (l0.get("title") or "").strip() + if not _BARE_AGREEMENT_PLAN_TITLE_RE.match(l0_title): + return rows + # Body presence is NOT a disqualifier: doc2dict's residual text-leaf + # body for the synthetic separator often contains the numbered + # sections' text. The body gets split out by + # `_split_l0_title_from_preamble` after the swap, into a regular + # L1 record. + + l0_node_id = l0["node_id"] + l0_penalty = l0.get("subdoc_penalty") or 0 + + # Step 2: find an earlier record whose title matches AGREEMENT|PLAN + # pattern with descriptive prefix (not bare single-word). + # `_STRUCTURAL_LEVELS` puts AGREEMENT|PLAN at level 0, so we use the + # same pattern for consistency. We pick the LAST (closest to the + # current L0) such record — for a multi-line title typeset as + # "SEPARATION AGREEMENT / AND / GENERAL RELEASE OF CLAIMS", only + # the FIRST line "SEPARATION AGREEMENT" ends in AGREEMENT, so that + # becomes the new L0. The other lines ("AND", "GENERAL RELEASE OF + # CLAIMS") get absorbed by `_merge_multiline_l0_title`'s forward + # walk (which is enabled by the `_swapped_l0` marker below). + _AGREEMENT_END_RE = re.compile( + r"^.*\b(AGREEMENT|PLAN)\s*$", + re.IGNORECASE, + ) + + real_title: dict[str, Any] | None = None + for r in rows: + if r["node_id"] >= l0_node_id: + break + if r.get("is_envelope") or r.get("scope") == "trailer": + continue + # Subdoc-penalty must match — we're looking for a sibling-equivalent + # title carrier in the same scope context (not buried inside a + # subdoc). + if (r.get("subdoc_penalty") or 0) != l0_penalty: + continue + title = (r.get("title") or "").strip() + if not title: + continue + # Must have descriptive prefix (longer than just "AGREEMENT"/"PLAN"). + if _BARE_AGREEMENT_PLAN_TITLE_RE.match(title): + continue + # Must match the AGREEMENT|PLAN end pattern. + if not _AGREEMENT_END_RE.match(title): + continue + real_title = r + # Don't break — pick the LAST (closest to current L0) such record. + + if real_title is None: + return rows + + # Step 3: swap depths. + # Promote real_title to L0; demote synthetic L0 to L1. + real_title["depth"] = 0 + l0_penalty + # Marker for `_merge_multiline_l0_title` to enable its FORWARD walk. + # Only the swap pathology requires a forward walk to collect + # title continuation lines AFTER the new L0; standard documents + # have all continuation lines BEFORE the L0 (the doc2dict-detected + # last-line). Without this marker the forward walk would + # over-absorb cover-preamble lines into the L0 title. + real_title["_swapped_l0"] = True + + # The synthetic L0 becomes L1 — it's a body-internal "AGREEMENT" + # separator that should still emit as source content. + l0["depth"] = 1 + l0_penalty + # Marker for `_explode_signature_block_lines` so the bare + # AGREEMENT/PLAN separator isn't claimed as a sig-block ancestor + # label by the UP-walk from /s/ carriers. Without this, the + # `_SIG_BLOCK_LABEL_RE` match against "AGREEMENT" (short uppercase + # title, empty body after the preamble split) would demote the + # separator to L2 in the signature area. + l0["_synthetic_l0_separator"] = True + + return rows + + def _merge_multiline_l0_title(rows: list[dict[str, Any]]) -> list[dict[str, Any]]: """Merge a multi-line agreement title into the single L0 record. @@ -2463,11 +2622,11 @@ def _merge_multiline_l0_title(rows: list[dict[str, Any]]) -> list[dict[str, Any] l0_pos = next(i for i, s in enumerate(siblings) if s["node_id"] == l0_node_id) except StopIteration: return rows - if l0_pos == 0: - return rows # L0 is the first sibling — no continuation lines. # Walk preceding siblings from L0 backwards. Collect a contiguous # run of title-continuation candidates ending immediately before L0. + # When L0 is the FIRST sibling (l0_pos==0), no preceding lines exist, + # but the forward walk below may still pick up trailing title lines. continuation: list[dict[str, Any]] = [] for i in range(l0_pos - 1, -1, -1): sib = siblings[i] @@ -2485,21 +2644,85 @@ def _merge_multiline_l0_title(rows: list[dict[str, Any]]) -> list[dict[str, Any] break continuation.append(sib) - if not continuation: + # Walk forward from L0 to collect continuation lines that follow the + # L0 title's first line. This is ONLY enabled when L0 was promoted + # by `_swap_synthetic_l0_with_real_title` (marked `_swapped_l0`). + # In that scenario, the L0 record is the FIRST line of a multi-line + # title; the trailing lines sit AFTER it as siblings. For example: + # "SEPARATION AGREEMENT
AND
GENERAL RELEASE OF CLAIMS" — + # doc2dict splits into three sibling records; after the swap, the + # first ("SEPARATION AGREEMENT") is L0, and the second ("AND") and + # third ("GENERAL RELEASE OF CLAIMS") follow. The third may carry + # the preamble body as body_direct — that body is preserved by + # transferring it to L0 (so the existing + # `_split_l0_title_from_preamble` can split it out as the L1 + # preamble record). + # + # Without the swap marker, the forward walk would over-absorb + # cover-preamble lines ("PANDORA MEDIA, INC.,", "and") that are + # NOT part of the title — they're cover-block content handled by + # `_rescue_cover_preamble_block` / `_rescue_post_title_cover_block`. + forward_continuation: list[dict[str, Any]] = [] + preamble_carrier: dict[str, Any] | None = None + if l0.get("_swapped_l0"): + for i in range(l0_pos + 1, len(siblings)): + sib = siblings[i] + if sib.get("is_envelope") or sib.get("scope") == "trailer": + break + if (sib.get("cls") or "") != "predicted header": + break + title = (sib.get("title") or "").strip() + body = (sib.get("body_direct") or "").strip() + # The sibling must look like a title continuation. + if not _looks_like_title_continuation(title): + break + # If this sibling carries body content, it's the FINAL title + # line — its body is the preamble that follows the title + # visually ("THIS SEPARATION AGREEMENT AND ... is entered + # into by ..."). We absorb its title into L0, transfer its + # body to L0 (so `_split_l0_title_from_preamble` can lift it + # as the L1 preamble), then stop the walk. + if body: + preamble_carrier = sib + forward_continuation.append(sib) + break + # Also stop the walk if this sibling has its own children + # — a title continuation line is always a leaf header. + if sib.get("n_direct_section_children", 0) > 0: + break + forward_continuation.append(sib) + + if not continuation and not forward_continuation: return rows # Build combined title in source order: earliest continuation line - # first, then L0's existing title. + # first, then L0's existing title, then forward continuations. continuation.reverse() # source order (oldest first) continuation_titles = [(c.get("title") or "").strip() for c in continuation] existing_l0_title = (l0.get("title") or "").strip() - combined = "\n".join(continuation_titles + [existing_l0_title]) + forward_titles = [(c.get("title") or "").strip() for c in forward_continuation] + combined = "\n".join(continuation_titles + [existing_l0_title] + forward_titles) l0["title"] = combined + # If a forward continuation carried the preamble body, transfer it to + # L0 so `_split_l0_title_from_preamble` will lift it to L1. + if preamble_carrier is not None: + carried_body = preamble_carrier.get("body_direct") or "" + l0["body_direct"] = carried_body + l0["body_direct_chars"] = len(carried_body) + # Preserve body_recursive too (used by some downstream probes). + if preamble_carrier.get("body_recursive"): + l0["body_recursive"] = preamble_carrier["body_recursive"] + l0["body_recursive_chars"] = len(l0["body_recursive"]) + preamble_carrier["body_direct"] = "" + preamble_carrier["body_direct_chars"] = 0 + # Mark the continuation records as envelope so they don't double-emit # to JSONL; parquet still records them. for c in continuation: c["is_envelope"] = True + for c in forward_continuation: + c["is_envelope"] = True return rows @@ -4450,6 +4673,13 @@ def _walk_descendants_local(nid: int) -> list[dict[str, Any]]: break if _has_section_marker_title(parent): break + # A synthetic body-internal AGREEMENT/PLAN separator (demoted + # from L0 by `_swap_synthetic_l0_with_real_title`) is NOT a + # sig-block party label — it's a content separator that + # happens to match the short-uppercase shape. Skip it. + if parent.get("_synthetic_l0_separator"): + cur = parent.get("parent_node_id") + continue p_title = (parent.get("title") or "").strip() p_body = (parent.get("body_direct") or "").strip() # Party-label ancestor: short uppercase title (strict @@ -4799,6 +5029,16 @@ def parse_one(idx: int, raw: str) -> tuple[dict[str, Any], list[dict[str, Any]]] # follow a numbered sibling into that numbered section so the tree # depths reflect the actual legal hierarchy. sections = _reparent_lettered_subsections_to_numbered_siblings(sections) + # If the current L0 is a body-internal section separator like + # "AGREEMENT" (a bare single-word centered bold paragraph that + # doc2dict's HTML walker mistakenly promoted to be the parent of the + # numbered body clauses), swap it with the earlier title carrier + # that ends in AGREEMENT/PLAN with a descriptive prefix + # ("SEPARATION AGREEMENT", "EMPLOYMENT AGREEMENT"). The earlier + # carrier becomes the real L0; the synthetic separator demotes to + # L1. Runs BEFORE the multi-line merge so the continuation walk + # operates on the correct L0. + sections = _swap_synthetic_l0_with_real_title(sections) # Merge multi-line agreement titles before splitting the preamble. # When the title is typeset across two visual lines (e.g. "AMENDMENT # NO.1 TO CRUDE TALL OIL AND\nBLACK LIQUOR SOAP SKIMMINGS AGREEMENT"), @@ -4806,7 +5046,10 @@ def parse_one(idx: int, raw: str) -> tuple[dict[str, Any], list[dict[str, Any]]] # the last line carries the AGREEMENT keyword and gets remapped to # depth=0; the upper line(s) sit at depth=1. This pass merges those # upper-line continuations into the L0 title before any downstream - # step uses the title for sorting or scope decisions. + # step uses the title for sorting or scope decisions. The merge also + # walks FORWARD when the swap pass picked an earlier first-line as + # L0, absorbing trailing continuation siblings (and a preamble- + # carrying final line) into the L0 title. sections = _merge_multiline_l0_title(sections) sections = _split_l0_title_from_preamble(sections) # Cover-preamble rescue: when the L0 title appears TWICE in the