Thread: [pgadmin-hackers] feature test timeouts

Thanks, patch applied.

On Fri, May 26, 2017 at 6:26 PM, George Gelashvili
<ggelashvili@pivotal.io> wrote:
> Hello Hackers,
>
> We are running tests including feature tests in Circle CI, and have been
> seeing some flakiness due to what we believe are insufficient wait-for
> timeouts.
>
> This shouldn't increase the overall test time, but only allow waiting longer
> for an element to appear, if needed.
>
> We've also split out the app start timeout since that seemed to need more
> time on CI.
>
> Patch attached.
>
> Thanks!
> Joao, Matt, and George
>
>
> --
> Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgadmin-hackers
>



--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


--
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers

Hi

OK. Do you know why? What's it waiting for? It was working for me (I
always run tests multiple times before committing them, to try to
ensure they're reliable).

On Mon, May 29, 2017 at 7:23 AM, Murtuza Zabuawala
<murtuza.zabuawala@enterprisedb.com> wrote:
> Hi Dave,
>
> With latest pull, PG data-type test fails on my system,
>
> ======================================================================
> ERROR: runTest
> (pgadmin.feature_tests.pg_datatype_validation_test.PGDataypeFeatureTest)
> Test checks for PG data-types output
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/feature_tests/pg_datatype_validation_test.py",
> line 44, in runTest
>     self._check_datatype()
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/feature_tests/pg_datatype_validation_test.py",
> line 97, in _check_datatype
>     self.page.fill_codemirror_area_with(query)
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 122, in fill_codemirror_area_with
>
> "//pre[contains(@class,'CodeMirror-line')]/../../../*[contains(@class,'CodeMirror-code')]").click()
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 87, in find_by_xpath
>     return self.wait_for_element(lambda driver:
> driver.find_element_by_xpath(xpath))
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 152, in wait_for_element
>     return self._wait_for("element to exist", element_if_it_exists)
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 188, in _wait_for
>     "Timed out waiting for " + waiting_for_message)
>   File
> "/Users/edb/Documents/projects/venv_pg4/lib/python3.5/site-packages/selenium/webdriver/support/wait.py",
> line 80, in until
>     raise TimeoutException(message, screen, stacktrace)
> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for
> element to exist
>
>
> ----------------------------------------------------------------------
> Ran 6 tests in 170.026s
>
> FAILED (errors=1)
>
> ======================================================================
> Test Result Summary
> ======================================================================
>
> PG_95:
>
> 5 tests passed
> 1 test failed:
> PGDataypeFeatureTest (Test checks for PG data-types output)
> 0 tests skipped
>
> ======================================================================
>
> Please check output in file:
> /Users/edb/Documents/projects/pgadmin4/web/regression/regression.log
>
>
> --
> Regards,
> Murtuza Zabuawala
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
> On Sat, May 27, 2017 at 11:47 PM, Dave Page <dpage@pgadmin.org> wrote:
>>
>> Thanks, patch applied.
>>
>> On Fri, May 26, 2017 at 6:26 PM, George Gelashvili
>> <ggelashvili@pivotal.io> wrote:
>> > Hello Hackers,
>> >
>> > We are running tests including feature tests in Circle CI, and have been
>> > seeing some flakiness due to what we believe are insufficient wait-for
>> > timeouts.
>> >
>> > This shouldn't increase the overall test time, but only allow waiting
>> > longer
>> > for an element to appear, if needed.
>> >
>> > We've also split out the app start timeout since that seemed to need
>> > more
>> > time on CI.
>> >
>> > Patch attached.
>> >
>> > Thanks!
>> > Joao, Matt, and George
>> >
>> >
>> > --
>> > Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
>> > To make changes to your subscription:
>> > http://www.postgresql.org/mailpref/pgadmin-hackers
>> >
>>
>>
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>>
>> --
>> Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgadmin-hackers
>
>



--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Hi Murtuza,

On Tue, May 30, 2017 at 4:13 PM, Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com> wrote:

Hi Dave,

I didn't look into the code for it but every time I run the tests it stays on below screen and fails. 

@Khushboo,

Any suggestion?

Harshal is also getting the same issue with his feature test case. The problem we found that, in the bound time the driver is not able to find the code-mirror. Me and Harshal looked into this yesterday but couldn't find the solution. 

BTW, I can not reproduce this issue at my end but me and Harshal will look into this issue on his machine.

Thanks,

Khushboo

 

--

Regards,

Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, May 30, 2017 at 2:15 PM, Dave Page <dpage@pgadmin.org> wrote:

Hi

OK. Do you know why? What's it waiting for? It was working for me (I
always run tests multiple times before committing them, to try to
ensure they're reliable).

On Mon, May 29, 2017 at 7:23 AM, Murtuza Zabuawala
<murtuza.zabuawala@enterprisedb.com> wrote:
> Hi Dave,
>
> With latest pull, PG data-type test fails on my system,
>
> ======================================================================
> ERROR: runTest
> (pgadmin.feature_tests.pg_datatype_validation_test.PGDataypeFeatureTest)
> Test checks for PG data-types output
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/feature_tests/pg_datatype_validation_test.py",
> line 44, in runTest
>     self._check_datatype()
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/feature_tests/pg_datatype_validation_test.py",
> line 97, in _check_datatype
>     self.page.fill_codemirror_area_with(query)
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 122, in fill_codemirror_area_with
>
> "//pre[contains(@class,'CodeMirror-line')]/../../../*[contains(@class,'CodeMirror-code')]").click()
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 87, in find_by_xpath
>     return self.wait_for_element(lambda driver:
> driver.find_element_by_xpath(xpath))
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 152, in wait_for_element
>     return self._wait_for("element to exist", element_if_it_exists)
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 188, in _wait_for
>     "Timed out waiting for " + waiting_for_message)
>   File
> "/Users/edb/Documents/projects/venv_pg4/lib/python3.5/site-packages/selenium/webdriver/support/wait.py",
> line 80, in until
>     raise TimeoutException(message, screen, stacktrace)
> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for
> element to exist
>
>
> ----------------------------------------------------------------------
> Ran 6 tests in 170.026s
>
> FAILED (errors=1)
>
> ======================================================================
> Test Result Summary
> ======================================================================
>
> PG_95:
>
> 5 tests passed
> 1 test failed:
> PGDataypeFeatureTest (Test checks for PG data-types output)
> 0 tests skipped
>
> ======================================================================
>
> Please check output in file:
> /Users/edb/Documents/projects/pgadmin4/web/regression/regression.log
>
>
> --
> Regards,
> Murtuza Zabuawala
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
> On Sat, May 27, 2017 at 11:47 PM, Dave Page <dpage@pgadmin.org> wrote:
>>
>> Thanks, patch applied.
>>
>> On Fri, May 26, 2017 at 6:26 PM, George Gelashvili
>> <ggelashvili@pivotal.io> wrote:
>> > Hello Hackers,
>> >
>> > We are running tests including feature tests in Circle CI, and have been
>> > seeing some flakiness due to what we believe are insufficient wait-for
>> > timeouts.
>> >
>> > This shouldn't increase the overall test time, but only allow waiting
>> > longer
>> > for an element to appear, if needed.
>> >
>> > We've also split out the app start timeout since that seemed to need
>> > more
>> > time on CI.
>> >
>> > Patch attached.
>> >
>> > Thanks!
>> > Joao, Matt, and George
>> >
>> >
>> > --
>> > Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
>> > To make changes to your subscription:
>> > http://www.postgresql.org/mailpref/pgadmin-hackers
>> >
>>
>>
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>>
>> --
>> Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgadmin-hackers
>
>



--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

We fixed the codemirror-filling helper function method thing in `pgadmin_page`. This should resolve the PG datatype feature test flakiness and make for a nice buttery crust.

It looked like selenium was unable to find an element within the sqleditor iframe, so we made a workaround that re-locates the iframe and codemirror element each iteration of the wait.

Thanks

Joao & George

On Tue, May 30, 2017 at 6:52 AM, Khushboo Vashi <khushboo.vashi@enterprisedb.com> wrote:

Hi Murtuza,

On Tue, May 30, 2017 at 4:13 PM, Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com> wrote:

Hi Dave,

I didn't look into the code for it but every time I run the tests it stays on below screen and fails. 

@Khushboo,

Any suggestion?

Harshal is also getting the same issue with his feature test case. The problem we found that, in the bound time the driver is not able to find the code-mirror. Me and Harshal looked into this yesterday but couldn't find the solution. 

BTW, I can not reproduce this issue at my end but me and Harshal will look into this issue on his machine.

Thanks,

Khushboo

 

--

Regards,

Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, May 30, 2017 at 2:15 PM, Dave Page <dpage@pgadmin.org> wrote:

Hi

OK. Do you know why? What's it waiting for? It was working for me (I
always run tests multiple times before committing them, to try to
ensure they're reliable).

On Mon, May 29, 2017 at 7:23 AM, Murtuza Zabuawala
<murtuza.zabuawala@enterprisedb.com> wrote:
> Hi Dave,
>
> With latest pull, PG data-type test fails on my system,
>
> ======================================================================
> ERROR: runTest
> (pgadmin.feature_tests.pg_datatype_validation_test.PGDataypeFeatureTest)
> Test checks for PG data-types output
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/feature_tests/pg_datatype_validation_test.py",
> line 44, in runTest
>     self._check_datatype()
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/feature_tests/pg_datatype_validation_test.py",
> line 97, in _check_datatype
>     self.page.fill_codemirror_area_with(query)
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 122, in fill_codemirror_area_with
>
> "//pre[contains(@class,'CodeMirror-line')]/../../../*[contains(@class,'CodeMirror-code')]").click()
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 87, in find_by_xpath
>     return self.wait_for_element(lambda driver:
> driver.find_element_by_xpath(xpath))
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 152, in wait_for_element
>     return self._wait_for("element to exist", element_if_it_exists)
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 188, in _wait_for
>     "Timed out waiting for " + waiting_for_message)
>   File
> "/Users/edb/Documents/projects/venv_pg4/lib/python3.5/site-packages/selenium/webdriver/support/wait.py",
> line 80, in until
>     raise TimeoutException(message, screen, stacktrace)
> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for
> element to exist
>
>
> ----------------------------------------------------------------------
> Ran 6 tests in 170.026s
>
> FAILED (errors=1)
>
> ======================================================================
> Test Result Summary
> ======================================================================
>
> PG_95:
>
> 5 tests passed
> 1 test failed:
> PGDataypeFeatureTest (Test checks for PG data-types output)
> 0 tests skipped
>
> ======================================================================
>
> Please check output in file:
> /Users/edb/Documents/projects/pgadmin4/web/regression/regression.log
>
>
> --
> Regards,
> Murtuza Zabuawala
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
> On Sat, May 27, 2017 at 11:47 PM, Dave Page <dpage@pgadmin.org> wrote:
>>
>> Thanks, patch applied.
>>
>> On Fri, May 26, 2017 at 6:26 PM, George Gelashvili
>> <ggelashvili@pivotal.io> wrote:
>> > Hello Hackers,
>> >
>> > We are running tests including feature tests in Circle CI, and have been
>> > seeing some flakiness due to what we believe are insufficient wait-for
>> > timeouts.
>> >
>> > This shouldn't increase the overall test time, but only allow waiting
>> > longer
>> > for an element to appear, if needed.
>> >
>> > We've also split out the app start timeout since that seemed to need
>> > more
>> > time on CI.
>> >
>> > Patch attached.
>> >
>> > Thanks!
>> > Joao, Matt, and George
>> >
>> >
>> > --
>> > Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
>> > To make changes to your subscription:
>> > http://www.postgresql.org/mailpref/pgadmin-hackers
>> >
>>
>>
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>>
>> --
>> Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgadmin-hackers
>
>



--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

We fixed the codemirror-filling helper function method thing in `pgadmin_page`. This should resolve the PG datatype feature test flakiness and make for a nice buttery crust.

It looked like selenium was unable to find an element within the sqleditor iframe, so we made a workaround that re-locates the iframe and codemirror element each iteration of the wait.

Thanks

Joao & George

On Tue, May 30, 2017 at 6:52 AM, Khushboo Vashi <khushboo.vashi@enterprisedb.com> wrote:

Hi Murtuza,

On Tue, May 30, 2017 at 4:13 PM, Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com> wrote:

Hi Dave,

I didn't look into the code for it but every time I run the tests it stays on below screen and fails. 

@Khushboo,

Any suggestion?

Harshal is also getting the same issue with his feature test case. The problem we found that, in the bound time the driver is not able to find the code-mirror. Me and Harshal looked into this yesterday but couldn't find the solution. 

BTW, I can not reproduce this issue at my end but me and Harshal will look into this issue on his machine.

Thanks,

Khushboo

 

--

Regards,

Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, May 30, 2017 at 2:15 PM, Dave Page <dpage@pgadmin.org> wrote:

Hi

OK. Do you know why? What's it waiting for? It was working for me (I
always run tests multiple times before committing them, to try to
ensure they're reliable).

On Mon, May 29, 2017 at 7:23 AM, Murtuza Zabuawala
<murtuza.zabuawala@enterprisedb.com> wrote:
> Hi Dave,
>
> With latest pull, PG data-type test fails on my system,
>
> ======================================================================
> ERROR: runTest
> (pgadmin.feature_tests.pg_datatype_validation_test.PGDataypeFeatureTest)
> Test checks for PG data-types output
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/feature_tests/pg_datatype_validation_test.py",
> line 44, in runTest
>     self._check_datatype()
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/feature_tests/pg_datatype_validation_test.py",
> line 97, in _check_datatype
>     self.page.fill_codemirror_area_with(query)
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 122, in fill_codemirror_area_with
>
> "//pre[contains(@class,'CodeMirror-line')]/../../../*[contains(@class,'CodeMirror-code')]").click()
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 87, in find_by_xpath
>     return self.wait_for_element(lambda driver:
> driver.find_element_by_xpath(xpath))
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 152, in wait_for_element
>     return self._wait_for("element to exist", element_if_it_exists)
>   File
> "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 188, in _wait_for
>     "Timed out waiting for " + waiting_for_message)
>   File
> "/Users/edb/Documents/projects/venv_pg4/lib/python3.5/site-packages/selenium/webdriver/support/wait.py",
> line 80, in until
>     raise TimeoutException(message, screen, stacktrace)
> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for
> element to exist
>
>
> ----------------------------------------------------------------------
> Ran 6 tests in 170.026s
>
> FAILED (errors=1)
>
> ======================================================================
> Test Result Summary
> ======================================================================
>
> PG_95:
>
> 5 tests passed
> 1 test failed:
> PGDataypeFeatureTest (Test checks for PG data-types output)
> 0 tests skipped
>
> ======================================================================
>
> Please check output in file:
> /Users/edb/Documents/projects/pgadmin4/web/regression/regression.log
>
>
> --
> Regards,
> Murtuza Zabuawala
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
> On Sat, May 27, 2017 at 11:47 PM, Dave Page <dpage@pgadmin.org> wrote:
>>
>> Thanks, patch applied.
>>
>> On Fri, May 26, 2017 at 6:26 PM, George Gelashvili
>> <ggelashvili@pivotal.io> wrote:
>> > Hello Hackers,
>> >
>> > We are running tests including feature tests in Circle CI, and have been
>> > seeing some flakiness due to what we believe are insufficient wait-for
>> > timeouts.
>> >
>> > This shouldn't increase the overall test time, but only allow waiting
>> > longer
>> > for an element to appear, if needed.
>> >
>> > We've also split out the app start timeout since that seemed to need
>> > more
>> > time on CI.
>> >
>> > Patch attached.
>> >
>> > Thanks!
>> > Joao, Matt, and George
>> >
>> >
>> > --
>> > Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
>> > To make changes to your subscription:
>> > http://www.postgresql.org/mailpref/pgadmin-hackers
>> >
>>
>>
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>>
>> --
>> Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgadmin-hackers
>
>



--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company