Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wip/detect boot #2

Open
wants to merge 255 commits into
base: master
Choose a base branch
from
Open

Wip/detect boot #2

wants to merge 255 commits into from

Conversation

jremmet
Copy link
Owner

@jremmet jremmet commented May 30, 2023

Description

Checklist

  • Documentation for the feature
  • Tests for the feature
  • The arguments and description in doc/configuration.rst have been updated
  • Add a section on how to use the feature to doc/usage.rst
  • Add a section on how to use the feature to doc/development.rst
  • PR has been tested
  • Man pages have been regenerated

@jremmet jremmet changed the base branch from master to origin/master May 30, 2023 13:07
@jremmet jremmet changed the base branch from origin/master to master May 30, 2023 13:07
@jremmet jremmet force-pushed the WIP/detect_boot branch from 2c330d7 to 04393a2 Compare May 30, 2023 13:17
@codecov-commenter
Copy link

codecov-commenter commented May 30, 2023

Codecov Report

Attention: Patch coverage is 55.18946% with 272 lines in your changes are missing coverage. Please review.

Project coverage is 53.2%. Comparing base (1318c27) to head (226b3eb).
Report is 255 commits behind head on master.

Files Patch % Lines
labgrid/driver/rawnetworkinterfacedriver.py 42.6% 51 Missing ⚠️
labgrid/driver/sigrokdriver.py 23.2% 43 Missing ⚠️
labgrid/driver/qemudriver.py 61.4% 27 Missing ⚠️
labgrid/resource/udev.py 6.8% 27 Missing ⚠️
labgrid/driver/sshdriver.py 59.2% 22 Missing ⚠️
labgrid/remote/client.py 5.2% 18 Missing ⚠️
labgrid/driver/power/eth008.py 27.2% 16 Missing ⚠️
labgrid/driver/powerdriver.py 5.8% 16 Missing ⚠️
labgrid/driver/power/poe_mib.py 29.4% 12 Missing ⚠️
labgrid/driver/bareboxdriver.py 65.2% 8 Missing ⚠️
... and 13 more

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff            @@
##           master      #2      +/-   ##
=========================================
- Coverage    63.2%   53.2%   -10.0%     
=========================================
  Files         152     163      +11     
  Lines       11330   12037     +707     
=========================================
- Hits         7169    6414     -755     
- Misses       4161    5623    +1462     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Bastian-Krause and others added 26 commits June 6, 2023 16:50
…rol/disk paths

When a udev event is retrieved from the UdevManager's queue, it is
matched against each resource. If a match is found, the resource's
device is updated (on "add"/"change"/"move" events) or removed (on
"unbind"/"remove" events).

Normally the `avail` attribute is set afterwards, but not for the
USBSDMuxDevice and USBSDWireDevice: those devices prevent their
USBResource super class from setting the `avail` attribute. Instead, the
devices are available depending on the availability of their disk and
control paths. Then the device's `update()` method is called, which is a
no-op for those devices.

On `poll()`, their disk and control paths is set to None if their device
is None. Otherwise, these paths are set if the device is not already
available.

In case of a USB reset (or fast replug) of a big USB tree, loads of udev
events need to be processed. Cases were observed where `poll()` did not
run between processing of a "remove" and "add" event of the same device.
This leads to a race condition, following the description from above:
The USBSDMuxDevice's or USBSDWireDevice's underlying device is set to
None on "remove" and then set to the new device again, on "add". The
`avail` attribute is not updated as intended. After that, the `poll()`
method runs: the device is set and the `avail` attribute evaluates to
True because the disk and control paths are still set from the time
before the reset replug. If this situation happens, these paths stay
invalid until the next reset/replug or restart of the labgrid exporter.

These invalid paths might point to non-existent /dev/sg* and dev/sd*
devices, rendering interaction with the Mux and its SD card impossible.
Or, even worse, the paths point to valid /dev/sg* and dev/sd* devices of
another Mux connected as observed here (note different USB devnums):

  $ labgrid-client -vv resources rlab/usb-2-p3/NetworkUSBSDMuxDevice
  Exporter 'rlab':
    Group 'usb-2-p3' (rlab/usb-2-p3/*):
      Resource 'USBSDMuxDevice' (rlab/usb-2-p3/NetworkUSBSDMuxDevice[/USBSDMuxDevice]):
        {'acquired': 'board1',
         'avail': True,
         'cls': 'NetworkUSBSDMuxDevice',
         'params': {'busnum': 2,
                    'control_path': '/dev/sg6',
                    'devnum': 42,
                    'extra': {'proxy': '[...]',
                              'proxy_required': False},
                    'host': 'rlab',
                    'model_id': 16449,
                    'path': '/dev/sdg',
                    'vendor_id': 1060}}
  $ labgrid-client -vv r rlab/usb-1-p4/NetworkUSBSDMuxDevice
  Exporter 'rlab':
    Group 'usb-1-p4' (rlabC-srv/c-usb-1-p4/*):
      Resource 'USBSDMuxDevice' (rlab/usb-1-p4/NetworkUSBSDMuxDevice[/USBSDMuxDevice]):
        {'acquired': 'board2',
         'avail': True,
         'cls': 'NetworkUSBSDMuxDevice',
         'params': {'busnum': 2,
                    'control_path': '/dev/sg6',
                    'devnum': 43,
                    'extra': {'proxy': '[...]',
                              'proxy_required': False},
                    'host': 'rlab',
                    'model_id': 16449,
                    'path': '/dev/sdg',
                    'vendor_id': 1060}}

Fix this race condition by implementing the disk and control paths reset
in the USBSDMuxDevice's or USBSDWireDevice's update() method, in case the
underlying device is gone. The update() method is called during the udev
event processing, so it does no longer matter when the poll() method is
called.

Fixes: 27ff02f ("udev: correct availability for USBSDMuxDevice")
Fixes: 9324d32 ("Add sdwire driver")
Signed-off-by: Bastian Krause <[email protected]>
…p-update

pyproject.toml: move pysnmp -> pysnmp-lextudio, drop unnecessary pysnmp-mibs
…mux-race

udev/resource: fix mux device race condition leading to outdated control/disk paths
Starting with version 6.1.0, the QEMU command line argument to enable
virtio with VirGL support was changed from "-vga virtio" to "-device
virtio-vgal-gl". To correctly handle this, scrape the QEMU version
number from the command output and use it to pass the correct arguments.

Signed-off-by: Joshua Watt <[email protected]>
doc: do not open if already closed -> open
Exposes the place tags in the RemotePlace. This allows persistent data
to be stored in the tags and retrieved by tests, for example unique MAC
address that need to be programmed to a device after flashing

Signed-off-by: Joshua Watt <[email protected]>
The option to use sftp was added multiple times when using
SSHDriver::put() and SSHDriver::get() causing them to only work one
time. Also adds the sftp option to SSHDriver::scp().

Signed-off-by: Nikolaj Rahbek <[email protected]>
No functional changes, a following commit will ensure that logging is
only configured when the logging plugin is available.

Signed-off-by: Rouven Czerwinski <[email protected]>
Check if the logging plugin is available and only configure logging if
it is. Fixes an internal pytest error during loading when the logging
plugin is explicitly disabled via "-p no:logging".

Signed-off-by: Rouven Czerwinski <[email protected]>
In case the logging plugin is not loaded, the option "log_cli_level" is
never registered and getoption fails. Catch the exception and return
from the function since there is nothing for us to configure in this
case. Fixes the following traceback:

  Traceback (most recent call last):
    File "/home/phoenix/work/ptx/labgrid/venv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1542, in getoption
      val = getattr(self.option, name)
  AttributeError: 'Namespace' object has no attribute 'log_cli_level'

  The above exception was the direct cause of the following exception:

  Traceback (most recent call last):
    File "/home/phoenix/work/ptx/labgrid/venv/bin/pytest", line 8, in <module>
      sys.exit(console_main())
    File "/home/phoenix/work/ptx/labgrid/venv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 190, in console_main
      code = main()
    File "/home/phoenix/work/ptx/labgrid/venv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 167, in main
      ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
    File "/home/phoenix/work/ptx/labgrid/venv/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
      return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
    File "/home/phoenix/work/ptx/labgrid/venv/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
      return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
    File "/home/phoenix/work/ptx/labgrid/venv/lib/python3.10/site-packages/pluggy/_callers.py", line 60, in _multicall
      return outcome.get_result()
    File "/home/phoenix/work/ptx/labgrid/venv/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
      raise ex[1].with_traceback(ex[2])
    File "/home/phoenix/work/ptx/labgrid/venv/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
      res = hook_impl.function(*args)
    File "/home/phoenix/work/ptx/labgrid/labgrid/pytestplugin/hooks.py", line 32, in pytest_cmdline_main
      set_cli_log_level(logging.DEBUG)
    File "/home/phoenix/work/ptx/labgrid/labgrid/pytestplugin/hooks.py", line 16, in set_cli_log_level
      current_level = config.getoption("log_cli_level") or config.getini("log_cli_level")
    File "/home/phoenix/work/ptx/labgrid/venv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1553, in getoption
      raise ValueError(f"no option named {name!r}") from e
  ValueError: no option named 'log_cli_level'

Signed-off-by: Rouven Czerwinski <[email protected]>
Add a test which uses the pytestplugin with -p no:logging set.

Signed-off-by: Rouven Czerwinski <[email protected]>
Fix handling disabled pytest logging plugin
…resource

Signed-off-by: Andreas Martinsson <[email protected]>
Signed-off-by: Jan Luebbe <[email protected]>
…bugger

Add SEGGER J-Link vendor and model IDs to USBDebugger resource
qemudriver: Handle Virtio GL change in 6.1.0
Signed-off-by: James Puderer <[email protected]>
Signed-off-by: Jan Luebbe <[email protected]>
This avoids the overhead on startup.

Signed-off-by: Jan Luebbe <[email protected]>
When using `power` command, one can use `-n` to choose to which resource
on a target the command will apply. However, when looping over resources
to find a compatible driver, it ignores the named resource. This will
cause an exception if another resource is chosen due another driver
being selected on the loop.

This patch fixes it by ignoring resources whose name do not match the
resource given in the command line.

Signed-off-by: Ederson de Souza <[email protected]>
A new NetworkYKUSHPowerPort is created to represent a remote ykush hub.
In order to control it via ssh, YKUSHPowerDriver now uses the `ykushcmd`
command instead of python module - it must be installed on the host
where the ykush is connected. One can get it from
https://github.com/Yepkit/ykush

A YKUSHPowerPortExport is also created to allow automatic exporting the
remote resource from a local one.

Signed-off-by: Chen Peng1 <[email protected]>
Signed-off-by: Ederson de Souza <[email protected]>
So that targets which expose YKUSHPowerPort instances can be controlled
via the command line client.

Signed-off-by: Ederson de Souza <[email protected]>
Bastian-Krause and others added 18 commits February 14, 2024 10:06
Other network resourced do not list the drivers using them, neither.

Signed-off-by: Bastian Krause <[email protected]>
Initially, all driver bindings were documented with their respective
binding key. This wasn't documented consistently since then.

Especially when using multiple instances of a single driver class, the
bindings are needed to specify on which resource or other driver the
driver should bind on. Until now, most binding keys had to be looked up
in the source code. So improve this situation by fixing and adding
binding keys as well as also adding all Remote/Network resource variants
a driver is able to bind to.

Signed-off-by: Bastian Krause <[email protected]>
Explicitly quote string top prevent strange yaml string auto dection
issues. Also use better readable dict notation everywhere.

Signed-off-by: Bastian Krause <[email protected]>
…yaml snippet

In the previous form, it looks like the environemnt's "drivers" section
also contains the "images" section, which is not the case. Make that
clear by splitting both sections into their own snippet.

Signed-off-by: Bastian Krause <[email protected]>
…tions-20240213

github/workflows: update GitHub actions to latest versions
Add USB ID for i.MX93 SoC to IMXUSBLoader.

Signed-off-by: Christian Hemp <[email protected]>
resource: udev: add new USB ID for IMXUSBLoader
The new major release 6.0 introduces breaking changes (functions are now
async), limit the dependency until we fix this properly.

Signed-off-by: Rouven Czerwinski <[email protected]>
pyproject.yaml: limit pysnmp to versions < 6
KarlK90 and others added 6 commits March 5, 2024 14:38
Decoding via VA-API fails for the C920 model and the playbin element on
AMD graphic cards, playbin3 has no problems though - so lets switch to
that.

Signed-off-by: Stefan Kerkmann <[email protected]>
…e-decoding

driver/usbvideodriver: use playbin3 element
…pdate

doc/configuration: update/fix bindings, protocols and resource usage
A USB serial device often provides multiple interfaces for the same
serial number. In these cases it is not possible to distinguish
the interfaces solely based on the serial number (ID_SERIAL_SHORT).
However, if additionally the interface number (ID_USB_INTERFACE_NUM)
is provided, it is possible to distinguish all interfaces without
the need of using the full device path (ID_PATH).

Example:

  (...)
  === suggested matches ===
  (...)
  USBSerialPort:
    match:
      ID_SERIAL_SHORT: ABCDEF00001
      ID_USB_INTERFACE_NUM: '00'
  (...)
  === suggested matches ===
  (...)
  USBSerialPort:
    match:
      ID_SERIAL_SHORT: ABCDEF00001
      ID_USB_INTERFACE_NUM: '01'

Signed-off-by: Joerg Hofrichter <[email protected]>
…t-interface-num

resource: udev: also suggest ID_USB_INTERFACE_NUM for USBResource
@jremmet jremmet force-pushed the WIP/detect_boot branch 2 times, most recently from 16bbdcd to 23c8e98 Compare March 25, 2024 15:25
jremmet added 2 commits March 26, 2024 08:34
Sets self.boot_detected in _await_prompt.
This is used in reset to check if a reboot has occurred.
Can also be used in strategies.

Signed-off-by: Jan Remmet <[email protected]>
Sets self.boot_detected in _await_prompt.
This is used in reset to check if a reboot has occurred.
Can also be used in strategies.

Direct checking was deprecated with:
7b55a19 ("driver/ubootdriver: deprecate boot_expression attribute")

Signed-off-by: Jan Remmet <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.