<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
- fixes: #5517
- fixes: #9250
For the following commands:
```
ls
# | le
| length
```
I found that it generates a bad lite parsing result:
```
LiteBlock {
block: [
LitePipeline {
commands: [
Command(None, LiteCommand { comments: [], parts: [Span { start: 138600, end: 138602 }] })
]
},
LitePipeline {
commands: [
Command(Some(Span { start: 138610, end: 138611 }),
LiteCommand { comments: [Span { start: 138603, end: 138609 }], parts: [Span { start: 138612, end: 138618 }] })
]
}
]
}
```
Which should contains only one `LitePipeline`, and the second
`LitePipeline` is generated because of `Eol` lex token:
```
[
Token { contents: Item, span: Span { start: 138600, end: 138602 } },
Token { contents: Eol, span: Span { start: 138602, end: 138603 } }, // it generates the second LitePipeline
Token { contents: Comment, span: Span { start: 138603, end: 138609 } },
Token { contents: Pipe, span: Span { start: 138610, end: 138611 } },
Token { contents: Item, span: Span { start: 138612, end: 138618 } }
]
```
To fix the issue, I remove the `Eol` token when we meet `Comment` right
after `Eol`, then it will generate a good LiteBlock, and everything will
work fine.
### After the fix:
Token:
```
[
Token { contents: Item, span: Span { start: 138618, end: 138620 } },
Token { contents: Comment, span: Span { start: 138622, end: 138628 } },
Token { contents: Pipe, span: Span { start: 138629, end: 138630 } },
Token { contents: Item, span: Span { start: 138631, end: 138637 } }
]
```
LiteBlock:
```
LiteBlock {
block: [
LitePipeline {
commands: [
Command(
None,
LiteCommand {
comments: [Span { start: 138622, end: 138628 }],
parts: [Span { start: 138618, end: 138620 }]
}
),
Command(
Some(Span { start: 138629, end: 138630 }),
LiteCommand { comments: [], parts: [Span { start: 138631, end: 138637 }] })] }] }
```
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect -A clippy::result_large_err` to check that
you're using the standard code style
- `cargo test --workspace` to check that all tests pass
- `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the
standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
90863439d1