Thread: Re: Schedule for migration to pglister
On Mon, Nov 06, 2017 at 10:36:38AM -0500, Stephen Frost wrote: > Our planned migration schedule is as follows: > > Nov 6 - > pgsql-www > > Nov 13 - > pgsql-hackers When each list migrated, its mbox archives stopped receiving new messages: https://www.postgresql.org/list/pgsql-www/mbox/pgsql-www.201711 https://www.postgresql.org/list/pgsql-hackers/mbox/pgsql-hackers.201711
Noah, * Noah Misch (noah@leadboat.com) wrote: > On Mon, Nov 06, 2017 at 10:36:38AM -0500, Stephen Frost wrote: > > Our planned migration schedule is as follows: > > > > Nov 6 - > > pgsql-www > > > > Nov 13 - > > pgsql-hackers > > When each list migrated, its mbox archives stopped receiving new messages: This was actually anticipated, though we had been thinking that the migration would be faster and so it wouldn't end up being such a long time that the mbox's wouldn't get updated. The plan is to replace those mbox's (which are created by mj2 and then rsync'd over currently) with ones generated from the archives database, but that's something which will basically happen to the entire site and all the lists at once. Further, there will actually be some differences between the archive-generated mbox's vs. those that came from mj2; in particular, the 'date' used by mj2 is the 'received' date (from what I can tell) while the 'date' in the archives database is the 'Date:' header from when the message is sent by the user. That said, this will make the mbox's match the actual webpages, which they don't currently because the webpages are built from the archive database while the mbox's are from mj2. In addition, while trying to cross-compare the two, I think I'm seeing some cases where mj2 is just outright dropping messages also (for example, I don't see https://www.postgresql.org/message-id/20170930224424.ud5ilchmclbl5y5n%40alap3.anarazel.de anywhere in the mj2 mboxes..). I'm chatting w/ Magnus about this now and I'm not sure exactly when we'll end up making the change, but my feeling is that we should either do it now, or on Monday when we migrate the 'user' lists. Doesn't seem to me like we should delay it any longer than that. Thanks! Stephen
Noah, * Noah Misch (noah@leadboat.com) wrote: > On Mon, Nov 06, 2017 at 10:36:38AM -0500, Stephen Frost wrote: > > Our planned migration schedule is as follows: > > > > Nov 6 - > > pgsql-www > > > > Nov 13 - > > pgsql-hackers > > When each list migrated, its mbox archives stopped receiving new messages: This was actually anticipated, though we had been thinking that the migration would be faster and so it wouldn't end up being such a long time that the mbox's wouldn't get updated. The plan is to replace those mbox's (which are created by mj2 and then rsync'd over currently) with ones generated from the archives database, but that's something which will basically happen to the entire site and all the lists at once. Further, there will actually be some differences between the archive-generated mbox's vs. those that came from mj2; in particular, the 'date' used by mj2 is the 'received' date (from what I can tell) while the 'date' in the archives database is the 'Date:' header from when the message is sent by the user. That said, this will make the mbox's match the actual webpages, which they don't currently because the webpages are built from the archive database while the mbox's are from mj2. In addition, while trying to cross-compare the two, I think I'm seeing some cases where mj2 is just outright dropping messages also (for example, I don't see https://www.postgresql.org/message-id/20170930224424.ud5ilchmclbl5y5n%40alap3.anarazel.de anywhere in the mj2 mboxes..). I'm chatting w/ Magnus about this now and I'm not sure exactly when we'll end up making the change, but my feeling is that we should either do it now, or on Monday when we migrate the 'user' lists. Doesn't seem to me like we should delay it any longer than that. Thanks! Stephen
On Wed, Nov 15, 2017 at 09:18:31AM -0500, Stephen Frost wrote: > * Noah Misch (noah@leadboat.com) wrote: > > When each list migrated, its mbox archives stopped receiving new messages: > > This was actually anticipated, though we had been thinking that the > migration would be faster and so it wouldn't end up being such a long > time that the mbox's wouldn't get updated. > > The plan is to replace those mbox's (which are created by mj2 and then > rsync'd over currently) with ones generated from the archives database, > but that's something which will basically happen to the entire site and > all the lists at once. > I'm chatting w/ Magnus about this now and I'm not sure exactly when > we'll end up making the change, but my feeling is that we should either > do it now, or on Monday when we migrate the 'user' lists. Doesn't seem > to me like we should delay it any longer than that. Gotcha. I, for one, can wait awhile.
On Wed, Nov 15, 2017 at 09:18:31AM -0500, Stephen Frost wrote: > * Noah Misch (noah@leadboat.com) wrote: > > When each list migrated, its mbox archives stopped receiving new messages: > > This was actually anticipated, though we had been thinking that the > migration would be faster and so it wouldn't end up being such a long > time that the mbox's wouldn't get updated. > > The plan is to replace those mbox's (which are created by mj2 and then > rsync'd over currently) with ones generated from the archives database, > but that's something which will basically happen to the entire site and > all the lists at once. > I'm chatting w/ Magnus about this now and I'm not sure exactly when > we'll end up making the change, but my feeling is that we should either > do it now, or on Monday when we migrate the 'user' lists. Doesn't seem > to me like we should delay it any longer than that. Gotcha. I, for one, can wait awhile.
On Wed, Nov 15, 2017 at 5:05 PM, Noah Misch <noah@leadboat.com> wrote:
On Wed, Nov 15, 2017 at 09:18:31AM -0500, Stephen Frost wrote:
> * Noah Misch (noah@leadboat.com) wrote:
> > When each list migrated, its mbox archives stopped receiving new messages:
>
> This was actually anticipated, though we had been thinking that the
> migration would be faster and so it wouldn't end up being such a long
> time that the mbox's wouldn't get updated.
>
> The plan is to replace those mbox's (which are created by mj2 and then
> rsync'd over currently) with ones generated from the archives database,
> but that's something which will basically happen to the entire site and
> all the lists at once.
> I'm chatting w/ Magnus about this now and I'm not sure exactly when
> we'll end up making the change, but my feeling is that we should either
> do it now, or on Monday when we migrate the 'user' lists. Doesn't seem
> to me like we should delay it any longer than that.
Gotcha. I, for one, can wait awhile.
On Wed, Nov 15, 2017 at 5:05 PM, Noah Misch <noah@leadboat.com> wrote:
On Wed, Nov 15, 2017 at 09:18:31AM -0500, Stephen Frost wrote:
> * Noah Misch (noah@leadboat.com) wrote:
> > When each list migrated, its mbox archives stopped receiving new messages:
>
> This was actually anticipated, though we had been thinking that the
> migration would be faster and so it wouldn't end up being such a long
> time that the mbox's wouldn't get updated.
>
> The plan is to replace those mbox's (which are created by mj2 and then
> rsync'd over currently) with ones generated from the archives database,
> but that's something which will basically happen to the entire site and
> all the lists at once.
> I'm chatting w/ Magnus about this now and I'm not sure exactly when
> we'll end up making the change, but my feeling is that we should either
> do it now, or on Monday when we migrate the 'user' lists. Doesn't seem
> to me like we should delay it any longer than that.
Gotcha. I, for one, can wait awhile.
Magnus, all, * Magnus Hagander (magnus@hagander.net) wrote: > mbox access should now be restored, and as Stephen says, now handled by the > Date in the message. Let me know if there are any issues with the new ones. > (They're now generated by the same code as the per-thread ones). And just fyi, that also means that complete threads end up in each mbox, so you'll see cases where a message was posted to, eg, -committers initially but the reply cc'd hackers, bringing the thread over there, and now the initial email to -committers and all of the subsequent emails on the thread will be included in the mbox for both -hackers and -committers. Hopefully everyone finds that to actually provide a better experience, but it's a difference to be aware of. Thanks! Stephen
Magnus, all, * Magnus Hagander (magnus@hagander.net) wrote: > mbox access should now be restored, and as Stephen says, now handled by the > Date in the message. Let me know if there are any issues with the new ones. > (They're now generated by the same code as the per-thread ones). And just fyi, that also means that complete threads end up in each mbox, so you'll see cases where a message was posted to, eg, -committers initially but the reply cc'd hackers, bringing the thread over there, and now the initial email to -committers and all of the subsequent emails on the thread will be included in the mbox for both -hackers and -committers. Hopefully everyone finds that to actually provide a better experience, but it's a difference to be aware of. Thanks! Stephen
On 2017-11-16 15:34, Stephen Frost wrote: > Magnus, all, > > * Magnus Hagander (magnus@hagander.net) wrote: >> mbox access should now be restored, and as Stephen says, now handled >> by the >> Date in the message. Let me know if there are any issues with the new >> ones. >> (They're now generated by the same code as the per-thread ones). > > And just fyi, that also means that complete threads end up in each > mbox, > so you'll see cases where a message was posted to, eg, -committers > initially but the reply cc'd hackers, bringing the thread over there, > and now the initial email to -committers and all of the subsequent > emails on the thread will be included in the mbox for both -hackers and > -committers. > > Hopefully everyone finds that to actually provide a better experience, > but it's a difference to be aware of. In theory :), that sounds better for people reading the web archives. Only one real way to find out "for sure" though, so fingers crossed. :) + Justin
On 2017-11-16 15:34, Stephen Frost wrote: > Magnus, all, > > * Magnus Hagander (magnus@hagander.net) wrote: >> mbox access should now be restored, and as Stephen says, now handled >> by the >> Date in the message. Let me know if there are any issues with the new >> ones. >> (They're now generated by the same code as the per-thread ones). > > And just fyi, that also means that complete threads end up in each > mbox, > so you'll see cases where a message was posted to, eg, -committers > initially but the reply cc'd hackers, bringing the thread over there, > and now the initial email to -committers and all of the subsequent > emails on the thread will be included in the mbox for both -hackers and > -committers. > > Hopefully everyone finds that to actually provide a better experience, > but it's a difference to be aware of. In theory :), that sounds better for people reading the web archives. Only one real way to find out "for sure" though, so fingers crossed. :) + Justin
On Thu, Nov 16, 2017 at 4:51 PM, Justin Clift <justin@postgresql.org> wrote:
On 2017-11-16 15:34, Stephen Frost wrote:Magnus, all,
* Magnus Hagander (magnus@hagander.net) wrote:mbox access should now be restored, and as Stephen says, now handled by the
Date in the message. Let me know if there are any issues with the new ones.
(They're now generated by the same code as the per-thread ones).
And just fyi, that also means that complete threads end up in each mbox,
so you'll see cases where a message was posted to, eg, -committers
initially but the reply cc'd hackers, bringing the thread over there,
and now the initial email to -committers and all of the subsequent
emails on the thread will be included in the mbox for both -hackers and
-committers.
Hopefully everyone finds that to actually provide a better experience,
but it's a difference to be aware of.
In theory :), that sounds better for people reading the web archives.
Um. Nothing has changed in the web interface, only the mbox downloads.
On Thu, Nov 16, 2017 at 4:51 PM, Justin Clift <justin@postgresql.org> wrote:
On 2017-11-16 15:34, Stephen Frost wrote:Magnus, all,
* Magnus Hagander (magnus@hagander.net) wrote:mbox access should now be restored, and as Stephen says, now handled by the
Date in the message. Let me know if there are any issues with the new ones.
(They're now generated by the same code as the per-thread ones).
And just fyi, that also means that complete threads end up in each mbox,
so you'll see cases where a message was posted to, eg, -committers
initially but the reply cc'd hackers, bringing the thread over there,
and now the initial email to -committers and all of the subsequent
emails on the thread will be included in the mbox for both -hackers and
-committers.
Hopefully everyone finds that to actually provide a better experience,
but it's a difference to be aware of.
In theory :), that sounds better for people reading the web archives.
Um. Nothing has changed in the web interface, only the mbox downloads.