Hello Folks,
Long time no see. Seems that I am back (for limited time) to Lava testing,
and after all the setups and catches 22, I managed to get to the bottom of
it, within few days.
I have interesting problem to report.
vagrant@stretch:/etc/lava-server/dispatcher-config/device-types$ dpkg -l
lava-server lava-dispatcher
Desired=Unknown/Install/Remove/Purge/Hold
|
Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version
Architecture Description
+++-==========================================-==========================-==========================-==========================================================================================
ii lava-dispatcher 2018.5-3~bpo9+1
amd64 Linaro Automated Validation Architecture
dispatcher
ii lava-server 2018.5-3~bpo9+1
all Linaro Automated Validation Architecture server
## Issue Background
Issue CIP testing #16 seems to be very similar: Beaglebone Black
health-check job is failing at restart
## Issue description
Wrong Ramdisk Image Format Ramdisk image is corrupt or invalid
## Acceptance criteria
The tftp 0x88080000 22/tftp-deploy-on2jld77/ramdisk/ramdisk.cpio.gz.uboot
Using cpsw devtftp 0x88080000
22/tftp-deploy-on2jld77/ramdisk/ramdisk.cpio.gz.uboot
The initramdisk is built by the following instructions:
https://wiki.linuxfoundation.org/civilinfrastructureplatform/cipsystembuild…
I used both BusyBox 28.0 and latest stable BusyBox 28.4 (failure seems to
be the same)!
Should download seamlessly, but it does not. It reports that the image is
corrupt. The full log is at:
local test of ramdisk test on bbb - Lava job 22
https://pastebin.com/y9n4GM5G
The .yaml file is at:
[lava 2018.5-3] job_name: local test of ramdisk test on bbb
https://pastebin.com/kqS2dqWM
_______
Namely, the download order is somehow scrambled!
Thank you,
Zoran Stojsavljevic
Hello
Since our upgrade to 2018.4 we experience lots of lava-logs crash with the
following trace in lava-logs.log
2018-06-20 13:43:08,964 INFO Saving 1 test cases
2018-06-20 13:43:16,614 DEBUG PING => master
2018-06-20 13:43:16,618 DEBUG master => PONG(20)
2018-06-20 13:43:19,524 INFO Saving 21 test cases
2018-06-20 13:43:29,535 INFO Saving 62 test cases
2018-06-20 13:43:37,983 DEBUG PING => master
2018-06-20 13:43:37,985 DEBUG master => PONG(20)
2018-06-20 13:43:39,541 INFO Saving 3 test cases
2018-06-20 13:43:58,009 DEBUG PING => master
2018-06-20 13:43:58,010 DEBUG master => PONG(20)
2018-06-20 13:44:01,770 INFO Saving 9 test cases
2018-06-20 13:44:01,771 ERROR [EXIT] Unknown exception raised, leaving!
2018-06-20 13:44:01,771 ERROR 'bool' object has no attribute 'pk'
Traceback (most recent call last):
File
"/usr/lib/python3/dist-packages/lava_server/management/commands/lava-logs.py",
line 181, in handle
self.main_loop()
File
"/usr/lib/python3/dist-packages/lava_server/management/commands/lava-logs.py",
line 232, in main_loop
self.flush_test_cases()
File
"/usr/lib/python3/dist-packages/lava_server/management/commands/lava-logs.py",
line 217, in flush_test_cases
TestCase.objects.bulk_create(self.test_cases)
File "/usr/lib/python3/dist-packages/django/db/models/manager.py", line
85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/lib/python3/dist-packages/django/db/models/query.py", line
441, in bulk_create
self._populate_pk_values(objs)
File "/usr/lib/python3/dist-packages/django/db/models/query.py", line
404, in _populate_pk_values
if obj.pk is None:
AttributeError: 'bool' object has no attribute 'pk'
2018-06-20 13:44:02,109 INFO [EXIT] Disconnect logging socket and
process messages
2018-06-20 13:44:02,109 DEBUG [EXIT] unbinding from 'tcp://0.0.0.0:5555'
2018-06-20 13:44:02,185 INFO Saving 9 test cases
2018-06-20 13:44:02,186 ERROR [EXIT] Unknown exception raised, leaving!
2018-06-20 13:44:02,186 ERROR 'bool' object has no attribute 'pk'
Traceback (most recent call last):
File
"/usr/lib/python3/dist-packages/lava_server/management/commands/lava-logs.py",
line 201, in handle
self.flush_test_cases()
File
"/usr/lib/python3/dist-packages/lava_server/management/commands/lava-logs.py",
line 217, in flush_test_cases
TestCase.objects.bulk_create(self.test_cases)
File "/usr/lib/python3/dist-packages/django/db/models/manager.py", line
85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/lib/python3/dist-packages/django/db/models/query.py", line
441, in bulk_create
self._populate_pk_values(objs)
File "/usr/lib/python3/dist-packages/django/db/models/query.py", line
404, in _populate_pk_values
if obj.pk is None:
AttributeError: 'bool' object has no attribute 'pk'
2018-06-20 13:44:02,186 INFO Saving 9 test cases
Any idea on how to fix this ?
Thanks
Regards
Hello everyone,
I have two cases in which I need to reboot my device during tests:
1. Reboot is active part of the test (e.g. store some persistent settings, reboot, check if persistent settings are correctly loaded after reboot)
2. Reboot is triggered and has to be evaluated (e.g. activate watchdog, stop resetting it, wait, check if system reboots automatically)
How can I hadle these two cases in LAVA?
Mit freundlichen Grüßen / Best regards
Tim Jaacks
DEVELOPMENT ENGINEER
Garz & Fricke GmbH
Tempowerkring 2
21079 Hamburg
Direct: +49 40 791 899 - 55
Fax: +49 40 791899 - 39
tim.jaacks(a)garz-fricke.com
www.garz-fricke.com<http://www.garz-fricke.com/>
SOLUTIONS THAT COMPLETE!
[cid:image001.jpg@01D407D7.E4232AA0]
Sitz der Gesellschaft: D-21079 Hamburg
Registergericht: Amtsgericht Hamburg, HRB 60514
Geschäftsführer: Matthias Fricke, Manfred Garz
Dear users,
the corresponding CVEs has been assigned:
* https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-12563
* https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-12564
* https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-12565
Regards
2018-06-15 23:29 GMT+02:00 Neil Williams <neil.williams(a)linaro.org>:
> 2018.5.post1
> ============
>
> During routine development, a new security scanning tool (bandit) was used
> on the LAVA codebase. Three security problems were found relating to the
> Job Submit UI and the loading of YAML files through XMLRPC. The problems
> date back to 2013, possibly earlier, so all releases of LAVA are affected.
>
> Fixes were developed and have now been released.
>
> https://review.linaro.org/#/c/25917/ Remove the ability to paste
> URLs in the submit page
>
> https://review.linaro.org/25918 Use requests instead of urlopen
>
> https://review.linaro.org/25919 Use yaml.safe_load when parsing
> user data
>
> Thanks to Remi Duraffort for identifying and fixing the issues.
>
> Note: These changes are not trivial to backport to previous releases. It
> is possible but some familiarity with the codebase will be required. We
> have packed a lot of changes into the time since the end of the migration
> and we are hoping to have a more stable time ahead. The LAVA software team
> recommend that all instances look to upgrade to 2018.5.post1. Our apologies
> for these problems.
>
> We are NOT aware of any exploits using these issues but now that the
> problems are public, it is prudent to apply the available fixes before
> anything happens.
>
> We expect to make more use of bandit and similar tools in future.
>
> CVE's have been requested but we don't have the CVE numbers back at this
> time.
>
> The production repo now carries these changes as 2018.5.post1-1+stretch
>
> An upload to Debian unstable will follow in due course. (The Debian
> security team were notified once we had a fix.) An upload to Debian
> Stretch to update 2016.12-1 is being prepared.
>
> --
>
> Neil Williams
> =============
> neil.williams(a)linaro.org
> http://www.linux.codehelp.co.uk/
>
> _______________________________________________
> Lava-announce mailing list
> Lava-announce(a)lists.linaro.org
> https://lists.linaro.org/mailman/listinfo/lava-announce
>
>
--
Rémi Duraffort
LAVA Team
Hi,
To match the result lines in the following log from zephyr sanity test:
— output —
***** Booting Zephyr OS v1.11.0-1194-g4b0b65c1b *****
Running test suite poll_api
===================================================================
starting test - test_poll_no_wait
PASS - test_poll_no_wait
===================================================================
starting test - test_poll_wait
PASS - test_poll_wait
===================================================================
starting test - test_poll_multi
PASS - test_poll_multi
===================================================================
===================================================================
— output ends —
I started with this pattern: '(?P<result>(PASS|FAIL))\s-\s(?P<test_case_id>\w+)', but the test_case_ids it matched are incomplete, shown as below. Refer to https://validation.linaro.org/scheduler/job/1807112
test_po
test_poll_
test_poll_mu
I also tried the following patterns, but no lucky.
'(?P<result>(PASS|FAIL))\s-\s(?P<test_case_id>\w+)$’ matched sth similar as above, but the not the same. Refer to https://validation.linaro.org/scheduler/job/1807117
'(?P<result>(PASS|FAIL))\s-\s(?P<test_case_id>\w+)\n’ didn’t match anything.
A search online hit https://stackoverflow.com/questions/14689531/how-to-match-a-new-line-charac… . Then I tried manually in python shell. '(?P<result>(PASS|FAIL))\s-\s(?P<test_case_id>\w+)’ works, '(?P<result>(PASS|FAIL))\s-\s(?P<test_case_id>\w+)$’ works only when re.M enabled.
— debug —
>>> s
"\nTrying ::1...\nConnected to localhost.\nEscape character is '^]'.\nFRDM-KW41Z-01 7113 [115200 N81]\n***** Booting Zephyr OS v1.11.0-1194-g4b0b65c1b *****\nRunning test suite poll_api\n===================================================================\nstarting test - test_poll_no_wait\nPASS - test_poll_no_wait\n===================================================================\nstarting test - test_poll_wait\nPASS - test_poll_wait\n===================================================================\nstarting test - test_poll_multi\nPASS - test_poll_multi\n===================================================================\n===================================================================\n"
>>> p.search(s).group()
'PASS - test_poll_no_wait'
>>> p = re.compile(r'(?P<result>(PASS|FAIL))\s-\s(?P<test_case_id>\w+)$')
>>> p.search(s).group()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'NoneType' object has no attribute 'group'
>>> p = re.compile(r'(?P<result>(PASS|FAIL))\s-\s(?P<test_case_id>\w+)$', re.M)
>>> p.search(s).group()
'PASS - test_poll_no_wait’
— ends —
Could you please advise me how to handle the parsing with the monitor action?
Thanks,
Chase
Good morning everyone,
I would like to know if the default password for lavaserver database
created in Postgresql is available somewhere in the default configuration
files?
Also, Is there a way to find out the default password for lavaserver user
in the host?
regards,
Hi all,
For the boards I am using in my LAVA lab, if I try an NFS job on my
jetson-tk1, it fails to mount the filesystem from the debian installed
NFS server.
http://lava.streamtester.net/scheduler/job/120050
my nfs-kernel-server version is 1:1.3.4-2.1, which was installed with
LAVA from Debian Stretch.
If I add 'vers=3' to the kernel NFS command line, it mounts the
filesystem successfully.
http://lava.streamtester.net/scheduler/job/120049
This is being discussed here to make it a default option
https://review.linaro.org/#/c/25666/
But really this does seem like there's an issue with the NFS kernel
server in Debian Stretch. Has anyone else had this issue?
Matt
Hello,
After upgrading to 2018.4 (also tried .5) many of our device-types
using base-uboot.jinja2 are broken. While I really like the major
improvement to run commands individually, there seems to be some
problems and the LAVA output logs are very confusing, showing
concatenated strings, etc.
Here is an example for an upstream device-type (meson-gxbb-p200), and
here is where it starts interacting with u-boot:
http://khilman.ddns.net/scheduler/job/15#L336
The "Parsed boot commands" look perfect, and all the commands in black
all look good, but notice the commands at the u-boot prompt, they
appear to be concatenated, starting right away at the "setenv
initrd_high ..."
However, observing the commands on the actual serial port (I use
conmux, so can observe the serial console interactions directly), I'm
not seeing concatenated strings, but the "setenv serverip ..." never
shows up, so the TFTP downloads fail, and the job fails.
Here's what I see directly on the serial console:
Hit Enter or space or Ctrl+C key to stop autoboot -- : 0
gxb_p200_v1#
gxb_p200_v1#setenv autoload no
gxb_p200_v1#setenv initrd_high 0xffffffff
gxb_p200_v1#setenv fdt_high 0xffffffff
gxb_p200_v1#dhcp
dwmac.c9410000 Waiting for PHY auto negotiation to complete.. done
Speed: 100, full duplex
BOOTP broadcast 1
BOOTP broadcast 2
DHCP client bound to address 192.168.0.216 (267 ms)
gxb_p200_v1#tftp 0x1080000 14/tftp-deploy-5v1wo7fv/kernel/uImage
Speed: 100, full duplex
Using dwmac.c9410000 device
TFTP from server 192.168.0.1; our IP address is 192.168.0.216
Filename '14/tftp-deploy-5v1wo7fv/kernel/uImage'.
Load address: 0x1080000
Loading: *
TFTP error: 'File not found' (1)
Even more interesting is that on the same setup, a beaglebone-black
device, using the same base-uboot.jinja2 is working just fine:
http://khilman.ddns.net/scheduler/job/1
Any help would be appreciated, I'm thoroughly confused by what's going on here.
Thanks,
Kevin
At some point last week - I think because of network connectivity issues
a job got stuck and I I cancelled it, it when run again it again appeared to hang. I again
cancelled it and am now seeing the health check not start (at least no
output appears on the job's webspage.
Looking at the output.yaml (in /var/lib/lava-server/default/media/job-output/2018/05/23/32 ) I see
... progress output for downloading https://images.validation.linaro.org/kvm/standard/stretch-2.img.gz
- {"dt": "2018-05-23T07:39:54.728015", "lvl": "debug", "msg": "[common] Preparing overlay tarball in /var/lib/lava/dispatcher/tmp/32/lava-overlay-aye3n2ke"}
- {"dt":
- "2018-05-23T07:39:54.728root@stretch:/var/lib/lava-server/default/media/job-output/2018/05/23/32
But none of this appears in http://localhost:8080/scheduler/job/32
and at the head of that page I see the message:
Unable to parse invalid logs: This is maybe a bug in LAVA that should be reported.
which other logs are best for checking whether this is an error that
should be fed back?
(LAVA 2018.4)
Robert
