This question from March 2020 points out that many questions/answers link to papers at the http://front.math.ucdavis.edu/ frontend instead of at the arXiv itself, but that the frontend had been down a few weeks. There are c.600 affected questions and it looks like a large proportion of the links could be fixed by a simple search-and-replace; so comments and the well-received answer suggested asking SE employees to do such a search-and-replace fix (moderators don’t have that ability), but felt “down for a few weeks” was too soon.

The frontend has now been down for 17 months; presumably this is no longer too soon. Can moderators now petition SE for a search-and-replace fix? Or if that’s too administratively difficult, should we just fix the links by hand, trying not to bump too many at once as usual?

Useful search queries, copied from the older question:

Search-and-replace suggestion: The links fall into several groups, some that should be easy to recognise and fix by regex, others less clear. Hopefully a large proportion can be fixed by regex, leaving the rest to fix by hand.

For links to preprints after April 2007, the correspondence is straightforward — we want to replace a url like http://front.math.ucdavis.edu/0809.1179 with https://arxiv.org/abs/0809.1179, and the first four digits represent YYMM. So a regex replacing https?://front.math.ucdavis.edu/([12]|0[8-9]|071|070[4-9]) with https://arxiv.org/abs/\1 should recognise and fix these links, I think. (Details depend on the regex flavour used, of course.)

Links to older preprints are less consistent. Some have the form https://front.math.ucdavis.edu/math.LO/9909115, and these should be easy again: the identifier part agrees with the arXiv’s, so a regex replace of https?://front.math.ucdavis.edu/(math[.a-zA-Z]*/\d+) by https://arxiv.org/abs/\1 should recognised and fix these. But others use a different identifier, whose correspondence with the arXiv’s identifiers seems unclear — e.g. http://front.math.ucdavis.edu/0310.5109 corresponds to https://arxiv.org/abs/math/0310109 (and https://arxiv.org/abs/0310.5109 doesn’t work). Does anyone else know or see how the correspondence works for these?

Finally there are links to category searches, and the like, which I don’t think we can hope to fix by regex. This edit history has examples of all three kinds.

  • $\begingroup$ You should add a suggested solution, e.g. a regex describing a good replacement strategy. $\endgroup$
    – Asaf Karagila Mod
    Aug 25 at 9:16
  • 3
    $\begingroup$ I will add that there are 248 comments with such links. (If Stack Exchange would be willing to use some automated proces to change such links, they can probably do so also in comments, not only in posts.) $\endgroup$ Aug 25 at 9:22
  • 1
    $\begingroup$ @AsafKaragila: I’ve added a regex suggestion which I think should get a large proportion of the broken links, and hopefully no false positives, although of course I haven’t been able to test/debug it on the full input set. $\endgroup$ Aug 25 at 9:43
  • 2
    $\begingroup$ Thanks! With the status-review, I hope we'll attract the attention of some StackOverlords soon. I'll see if I can speed it up as well. $\endgroup$
    – Asaf Karagila Mod
    Aug 25 at 9:51
  • $\begingroup$ @PeterLeFanuLumsdaine Are you sure that this revision history is an example you want? This seems to be link to a search rather than a link to and article in the "old format". I left a few more comment in MO editors' lounge. $\endgroup$ Aug 25 at 10:07
  • $\begingroup$ @MartinSleziak: The edit diff in that link shows three different kinds of example: one search link, which I don’t think we could hope to fix by regex; two of the post-April-2007 links that my suggested regex should fix; and two of the pre-April-2007 links that might or might not be regex-able. $\endgroup$ Aug 25 at 10:32
  • $\begingroup$ My mistake. I thought that by "pre-2007" you mean links of the form similar to https://front.math.ucdavis.edu/math.LO/9909115. $\endgroup$ Aug 25 at 10:36
  • 1
    $\begingroup$ @MartinSleziak: Right — the confusing thing is that for pre-2007 papers, the frontend seems to provide url’s in at least two different forms, some like the example you give, some like the ones in the edit history I linked. For the ones in the edit history, the correspondence with arXiv’s url/identifier is totally unclear to me. For examples like your form, the conversion is clearer — the identifier math.LO agrees with the arXiv, so we just need to change the url prefix — but it would take a little thought to write a regex that’ll recognise url’s of that form without false positives. $\endgroup$ Aug 25 at 11:23
  • 2
    $\begingroup$ @Martin: The [status-review] puts the thing on their todo list, which is probably long enough that it will take a while. We absolutely should figure out the algorithm for replacing until then, though. $\endgroup$
    – Asaf Karagila Mod
    Aug 25 at 11:30
  • 2
    $\begingroup$ Maybe for some of them it is removing .5 and adding /math and replacing .47 by 0 and adding /math-ph? (But I only tested it on a few URLs.) $\endgroup$ Aug 25 at 12:20
  • 1
    $\begingroup$ Shouldn’t the regex for post-2007 include numbers starting with 2 as well? $\endgroup$ Aug 25 at 12:22
  • 2
    $\begingroup$ @EmilJeřábek: Thanks, fixed! @ Martin: yes, from a few examples it looks like the system works something like that. But at least for my part, I wouldn’t feel confident proposing a regex for a half-guessed system without being able to test the regex out a bit. If you feel confident enough to add a regex to handle these then that’d be great! $\endgroup$ Aug 25 at 13:59
  • 1
    $\begingroup$ Does somebody know whether lanl.arxiv.org is down only temporarily or whether all links to that domain should be considered dead, too? (This is not directly related to the front.math.ucdavis.edu, but it seems like a similar problem, so I thought that it might be worth mentioning.) $\endgroup$ Sep 11 at 8:03
  • 1
    $\begingroup$ Just to note, Martin and I found yet another oddity. The arXiv front identifier "9612.5114" which I thought should be "math/9612114", really points to "dg-ga/9612014". So maybe ***.51** = dg-ga? $\endgroup$ Sep 23 at 3:27
  • 1
    $\begingroup$ Another weird data point. I think the link front.math.ucdavis.edu/9712.5188 should point to arxiv.org/abs/q-alg/9712048 $\endgroup$ Oct 5 at 3:03

I've reviewed your request and - while I'd generally want to say "yes, we can do this" - in this case, I'm sorry to say that I don't think it will be possible - at least, not without some effort from you all first. I really do think it's important that the links on this site are functional, so I appreciate your bringing this up for discussion.

The tool we have to make this change is a pretty simple find/replace tool that only works for post content, not comments. It only works for URLs and it can't accept RegEx. So, I think you may already see the problem. While I really appreciate the effort that y'all went to to find the various treatments of the URLs, we can't even only update the ones that are an easy swap without changing all of the others since the content needing to be replaced is the same in all cases.

As you might also see, this doesn't mean that we can't ever do this - it means that we'd need you to help us out before we can. Based on the comments, it sounds like about 2/3 of the ~900 links are the "easy" first and second type. If that's the case, you could manually repair the remaining ~300 URLs first and we can use the link editing tool to fix the remainder of the links which, despite being in two formats require the same change to happen since arXiv helpfully automatically redirects to the correct URL.

So to put it in other words:

  1. First you all work together to fix the outstanding links of the second and third type manually.

    • Based on a comment from Martin Sleziak it looks like there's about 300 links in the third category. It seems like correcting these links in an automated way may be difficult regardless since it's unclear how they should change.
  2. Once the difficult URLs are fixed, we will bulk find/replace the remaining using the following method:

    • Find http://front.math.ucdavis.edu/ and replace with https://arxiv.org/abs/

As far as I understand, we can not bulk edit all 900 URLs first as it will make the pre-2007 URLs even more "wrong" and may make them more difficult to find and repair at a later point in time.

If you can find a consistent way to convert from one of the pre-2007 URLs to the new arXiv URL, you may be able to have someone create a userscript to update these over time, a few questions per day but that's a big "if".

Please let me know if this is clear or if you have any other questions about this response.

  • $\begingroup$ As someone whose edit history will show his endless appetite for tidying, I would be happy to be part of such a manual effort, which presumably it would be unobjectionable to spread over time. However, is there any way to do it without meaninglessly promoting hundreds of posts to the front page? $\endgroup$
    – LSpice
    Sep 14 at 3:28
  • $\begingroup$ @LSpice Something like that was discussed several times, you can see various feature requests around this. (I still hope that this is going to be implemented at some point - but AFAICT, at the moment there is no way to edit posts without bumping.) See also: Minor edits, subject to review (on this meta) or Allow non-bumping minor edits, but review them on /review (on Meta Stack Exchange). $\endgroup$ Sep 14 at 5:10
  • 1
    $\begingroup$ The tool we have to make this change is a pretty simple find/replace tool that only works for post content, not comments. It only works for URLs and it can't accept RegEx. You are a software company, so making new tools to interact with your main product should be a normal thing for you. And a regex find/replace engine is something that could have many other applications in additions to this task. I imagine it there are prioritization and cost/benefit issues here; "we don't have the tools to do that" surely is not the whole story. $\endgroup$ Sep 16 at 10:18
  • 1
    $\begingroup$ @FedericoPoloni considering the vast list of things that need to be addressed, building a better tool for this specific issue is extremely low priority. $\endgroup$
    – Catija StaffMod
    Sep 16 at 12:18
  • $\begingroup$ @LSpice I've been attacking them slowly, using the search tools in Martin's answer. It would be nice to divide the effort. Perhaps attacking the "Maybe manually, maybe mass converted" ones would be a good place to start, they could be knocked off over a week or so. The pre-2000 ones likewise. The ~200 posts with links from 2000–06 are going to be a slow slog. There may be some overlap, though, which will help—I've seen posts with multiple links from all over the shop. $\endgroup$ Sep 23 at 2:13
  • $\begingroup$ @DavidRoberts, I definitely still want to help, although, as for everyone, the semester being underway cuts down dramatically on my MO time (hence my near-two weeks' delay). Have you just been editing them as regular posts, in a way that bumps them to the front page? (I understand that that is the norm, and know no better way, but wanted to make sure it was the appropriate way to proceed before diving in.) $\endgroup$
    – LSpice
    Oct 4 at 17:06
  • 1
    $\begingroup$ @LSpice yes, I've been just doing a couple at a time, the normal way. Even just attacking one a day would be progress. The posts are spread over a wide range of topics, so it's not like fixing up a whole bunch on a niche topic and crowding the front page. $\endgroup$ Oct 4 at 23:06
  • $\begingroup$ Meanwhile, the tool was apparently significantly improved. Does this change the answer in any way? $\endgroup$ Oct 7 at 14:55
  • $\begingroup$ @EmilJeřábek No, it doesn't. The improvements made don't include regex - the main improvement was that we now can run the query across the network rather than per-site. CMs also have the ability to preview the impact of the changes before requesting the dev team makes them. Regex hasn't been added. $\endgroup$
    – Catija StaffMod
    Oct 7 at 15:03
  • $\begingroup$ All right, thanks for the explanation. $\endgroup$ Oct 7 at 15:24

At this moment, it is unclear whether a project like this should be done. (After all, until we heard from people running the arXiv frontend, we still should count as a possibility that it will be running again at some point.)

Still, it might be useful to analyze a bit what would the scope of the project be - how many links would need manual editing, how many links can be mass-converted, etc.

I have started this CW answer - please do edit it if you have additional information that might be useful here.


Notice that the numbers listed below are about the status at the time when this answer was first posted - you can find how the counts have changed using the links given below and from the updates added to this post.

Links which can be mass-converted

Links which have to be edited manually

(DR: I claim all the rest are dealt with.)

  • Ids starting with 9 are before 2000. At the moment, we get 41 posts from search, and from SEDE 43 posts and 12 comments (DR: all of these comments have a reply pointing out the updated link and title of the paper) SEDE - last activity, SEDE - already edited

  • Ids starting with 0700 to 0703 are also before April 2007. SEDE returns 3 comments (DR: I have replied to these comments with updated links, paper titles and doi where available) and 8 posts. SEDE - already edited

  • The above includes also the link http://front.math.ucdavis.edu/0208.5107v2 from this answer which doesn't exactly fit the pattern. This seems to be the only instance of linking to the version number.

  • The few instances where front.math.ucdavis.edu is used as a plain text (i.e., it is following neither http:// nor https://) should be treated on case-by-case basis. SEDE: posts. (DR: there is one remaining text instance that is not intended as a link, but as a mention of the arxiv front.) There are also 9 such comments. (DR: I have gone through and replied to the offending comments with updated links in all cases, and added paper titles and also doi links to the published versions where available) SEDE - already edited

  • There is only one post with front.math.ucdavis.edu/abs/ (search, SEDE) - a single post can be dealt with manually.

  • There are only a few posts with front.math.ucdavis.edu/q-alg/ - they can be dealt with manually. (A few have already been edited.)

Maybe manually, maybe mass converted

Links with math.AG, math.AT, math.LO etc.: They probably can be converted automatically, but a separate replacement for each category would be needed. Considering the low count, maybe manual editing might be preferable. SEDE returns 27 posts (DR: these have all been manually fixed, I took the opportunity to add more value by giving paper details and links to published versions and also fix other nearby broken links) and 6 comments (DR: all these comments have a reply giving the updated link and title of the paper.) SEDE - last activity, SEDE - already edited

Links not going to articles

  • 1
    $\begingroup$ I have struck of a couple that needed only two post edits. We should be able to get some of the items here, with only a few posts each, cleaned up within a couple of days, then likewise clean up the list. $\endgroup$ Sep 20 at 5:30

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .