Orbi WiFi 7 RBE973

BR500 Connectivity Issue - Subscribe Here for Updates

BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

Message 51 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

10PM , 11/16/2019 PST.

There was big connection drop right after each proxy dropped some connections at same time (10:11AM), average 15 connections per server,

Total dropped from 580 to 460. We are trying to find out root cause.  A war room is ongoing.

 

Message 52 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

10AM , 11/17/2019 PST.

All servers operational. Total connection number was 584.

CloudOPS to perform integrity check before working hours.

Message 53 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

Message 54 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

11AM , 11/18/2019 PST.

All servers operational. Total connections is 560 which is 30 less than peak connection 590+.

Some users might have been hit by conductor server 02 incident but have not reported/found issue yet.

Proxy server 01 and 02 don’t seem to be 100% healthy, they contributed to bulk of 30 connection drop.

 

Region

11AM

-0.5Day

-1D

-1.5D

-2D

-2.5D

-3D

1: US-W

76

84

84

82

87

87

88

2: US-E

58

64

56

57

77

77

81

3: EU

64

66

77

81

69

69

71

4: US-E

53

54

56

50

62

62

63

5: US-W

87

83

86

85

67

65

71

6: US-W

89

86

90

94

97

96

95

7: EU

71

70

65

64

60

61

61

8: US-E

62

61

69

74

61

59

64

Total

560

568

583

587

580

576

594

 

Message 55 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

10PM , 11/18/2019 PST.

All servers operational. No VPN issue reported from customer.

 

Region

10PM

-0.5Day

-1Day

-1.5D

-2D

-2.5D

-3D

1: US-W

77

76

84

84

82

87

87

2: US-E

58

58

64

56

57

77

77

3: EU

58

64

66

77

81

69

69

4: US-E

52

53

54

56

50

62

62

5: US-W

86

87

83

86

85

67

65

6: US-W

88

89

86

90

94

97

96

7: EU

72

71

70

65

64

60

61

8: US-E

61

62

61

69

74

61

59

Total

552

560

568

583

587

580

576

Message 56 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

11AM , 11/20/2019 PST.

All servers operational. Although no new case reported, few users are still affected by VPN group info issue. Trying to unblock all affected users ASAP.

 

Region

Current

-0.5Day

-1D

-1.5D

-2D

-2.5D

-3D

1: US-W

61

70

83

77

76

84

84

2: US-E

57

53

56

58

58

64

56

3: EU

80

62

53

58

64

66

77

4: US-E

50

51

50

52

53

54

56

5: US-W

86

88

94

86

87

83

86

6: US-W

85

80

88

88

89

86

90

7: EU

69

67

69

72

71

70

65

8: US-E

69

65

65

61

62

61

69

Total

557

536

558

552

560

568

583

Message 57 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

Message 58 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

Message 59 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

Message 60 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

10AM, 11/22/2019 PST.

 

External notes:

  • All servers operational.
  • One user reported that VPN user may not see all available VPN group(s), this is a known issue that we are trying to unblock all affected users as soon as possible.

 

Region

Current

-0.5Day

-1D

-1.5D

-2D

-2.5D

-3D

1: US-W

41

42

50

52

61

70

83

2: US-E

68

69

65

58

57

53

56

3: EU

88

82

84

76

80

62

53

4: US-E

45

51

52

51

50

51

50

5: US-W

86

86

83

87

86

88

94

6: US-W

72

79

83

85

85

80

88

7: EU

61

67

68

66

69

67

69

8: US-E

69

64

64

65

69

65

65

Total

530

540

549

540

557

536

558

Message 61 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

Message 62 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

Message 63 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

Message 64 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

Message 65 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

Message 66 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

1:45 am , 11/25/2019 PST.

 

Update

  • One of the proxy server in the Ireland region went down at 12:46 am PST and got recovered at 1:20 am.
  • somewhere total of 97 connection would have been impacted.
  • All servers operational now.

Recommendation :

Most user’s VPN connection should have recovered by itself.

In rare cases, VPN service might need to be updated with correct info(if user’s site to site is OK, but client can’t join the VPN group).

This can be done though following method:

 

    Recommended method: User login GUI page: https://ip-of-router/support.html switch to any other proxy server

    Or move router out of and add back to VPN group

    Or restart VPN service though SDM connection

    Or restart router

 

 

Region

current

Current

-0.5Day

-1D

-1.5D

-2D

-2.5D

-3D

1: US-W

37

37

36

37

35

36

41

42

2: US-E

73

70

68

67

68

68

68

69

3: EU

85

91

91

91

88

89

88

82

4: US-E

49

47

47

45

47

49

45

51

5: US-W

91

89

89

89

89

88

86

86

6: US-W

76

77

78

79

81

75

72

79

7: EU

63

59

58

59

59

62

61

67

8: US-E

70

67

67

67

71

68

69

64

Total

544

537

534

534

538

535

530

540

Message 67 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

10AM , 11/25/2019 PST.

 

External notes:

  • All servers operational.
  • One of Europe VPN server was restarted at 1:20AM PST, 11/15. VPN connections facilitated by this server may experience connection issue between 12:46AM and 1:20AM PST, 11/25/19

 

Region

10AM

1:45AM

-0.5Day

1D

-1.5D

-2D

-2.5D

-3D

1: US-W

38

37

37

36

37

35

36

41

2: US-E

71

73

70

68

67

68

68

68

3: EU

86

85

91

91

91

88

89

88

4: US-E

49

49

47

47

45

47

49

45

5: US-W

92

91

89

89

89

89

88

86

6: US-W

80

76

77

78

79

81

75

72

7: EU

66

63

59

58

59

59

62

61

8: US-E

70

70

67

67

67

71

68

69

Total

552

544

537

534

534

538

535

530

Message 68 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

Message 69 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

1PM , 11/26/2019 PST.

 External notes:

  • All servers operational.
  • Some users were not able to delete VPN group, the issue should have been fixed. All VPN group operations should be functional, please contact our support if issues persist.
  • Several VPN users don’t see all available VPN groups, we are working on the issue.

 

Region

Current

-0.5Day

-1D

-1.5D

-2D

-2.5D

-3D

1: US-W

45

36

38

37

36

37

35

2: US-E

68

69

71

70

68

67

68

3: EU

98

83

86

91

91

91

88

4: US-E

51

46

49

47

47

45

47

5: US-W

89

93

92

89

89

89

89

6: US-W

89

78

80

77

78

79

81

7: EU

66

65

66

59

58

59

59

8: US-E

68

69

70

67

67

67

71

Total

574

539

552

537

534

534

538

Message 70 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

Message 71 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

Message 72 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

Message 73 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

Message 74 of 295
BretD
Admin

Re: BR500 Connectivity Issue - Subscribe Here for Updates

Message 75 of 295
Top Contributors
Discussion stats
  • 294 replies
  • 25020 views
  • 7 kudos
  • 1 in conversation
Announcements

Need More Help?