Commit 834f8933 authored by Vladimir Oltean's avatar Vladimir Oltean Committed by David S. Miller

net: dsa: sja1105: implement tc-gate using time-triggered virtual links

Restrict the TTEthernet hardware support on this switch to operate as
closely as possible to IEEE 802.1Qci as possible. This means that it can
perform PTP-time-based ingress admission control on streams identified
by {DMAC, VID, PCP}, which is useful when trying to ensure the
determinism of traffic scheduled via IEEE 802.1Qbv.

The oddity comes from the fact that in hardware (and in TTEthernet at
large), virtual links always need a full-blown action, including not
only the type of policing, but also the list of destination ports. So in
practice, a single tc-gate action will result in all packets getting
dropped. Additional actions (either "trap" or "redirect") need to be
specified in the same filter rule such that the conforming packets are
actually forwarded somewhere.

Apart from the VL Lookup, Policing and Forwarding tables which need to
be programmed for each flow (virtual link), the Schedule engine also
needs to be told to open/close the admission gates for each individual
virtual link. A fairly accurate (and detailed) description of how that
works is already present in sja1105_tas.c, since it is already used to
trigger the egress gates for the tc-taprio offload (IEEE 802.1Qbv). Key
point here, we remember that the schedule engine supports 8
"subschedules" (execution threads that iterate through the global
schedule in parallel, and that no 2 hardware threads must execute a
schedule entry at the same time). For tc-taprio, each egress port used
one of these 8 subschedules, leaving a total of 4 subschedules unused.
In principle we could have allocated 1 subschedule for the tc-gate
offload of each ingress port, but actually the schedules of all virtual
links installed on each ingress port would have needed to be merged
together, before they could have been programmed to hardware. So
simplify our life and just merge the entire tc-gate configuration, for
all virtual links on all ingress ports, into a single subschedule. Be
sure to check that against the usual hardware scheduling conflicts, and
program it to hardware alongside any tc-taprio subschedule that may be
present.

The following scenarios were tested:

1. Quantitative testing:

   tc qdisc add dev swp2 clsact
   tc filter add dev swp2 ingress flower skip_sw \
           dst_mac 42:be:24:9b:76:20 \
           action gate index 1 base-time 0 \
           sched-entry OPEN 1200 -1 -1 \
           sched-entry CLOSE 1200 -1 -1 \
           action trap

   ping 192.168.1.2 -f
   PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data.
   .............................
   --- 192.168.1.2 ping statistics ---
   948 packets transmitted, 467 received, 50.7384% packet loss, time 9671ms

2. Qualitative testing (with a phase-aligned schedule - the clocks are
   synchronized by ptp4l, not shown here):

   Receiver (sja1105):

   tc qdisc add dev swp2 clsact
   now=$(phc_ctl /dev/ptp1 get | awk '/clock time is/ {print $5}') && \
           sec=$(echo $now | awk -F. '{print $1}') && \
           base_time="$(((sec + 2) * 1000000000))" && \
           echo "base time ${base_time}"
   tc filter add dev swp2 ingress flower skip_sw \
           dst_mac 42:be:24:9b:76:20 \
           action gate base-time ${base_time} \
           sched-entry OPEN  60000 -1 -1 \
           sched-entry CLOSE 40000 -1 -1 \
           action trap

   Sender (enetc):
   now=$(phc_ctl /dev/ptp0 get | awk '/clock time is/ {print $5}') && \
           sec=$(echo $now | awk -F. '{print $1}') && \
           base_time="$(((sec + 2) * 1000000000))" && \
           echo "base time ${base_time}"
   tc qdisc add dev eno0 parent root taprio \
           num_tc 8 \
           map 0 1 2 3 4 5 6 7 \
           queues 1@0 1@1 1@2 1@3 1@4 1@5 1@6 1@7 \
           base-time ${base_time} \
           sched-entry S 01  50000 \
           sched-entry S 00  50000 \
           flags 2

   ping -A 192.168.1.1
   PING 192.168.1.1 (192.168.1.1): 56 data bytes
   ...
   ^C
   --- 192.168.1.1 ping statistics ---
   1425 packets transmitted, 1424 packets received, 0% packet loss
   round-trip min/avg/max = 0.322/0.361/0.990 ms

   And just for comparison, with the tc-taprio schedule deleted:

   ping -A 192.168.1.1
   PING 192.168.1.1 (192.168.1.1): 56 data bytes
   ...
   ^C
   --- 192.168.1.1 ping statistics ---
   33 packets transmitted, 19 packets received, 42% packet loss
   round-trip min/avg/max = 0.336/0.464/0.597 ms
Signed-off-by: default avatarVladimir Oltean <vladimir.oltean@nxp.com>
Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
parent dfacc5a2
......@@ -36,6 +36,7 @@ struct sja1105_regs {
u64 status;
u64 port_control;
u64 rgu;
u64 vl_status;
u64 config;
u64 sgmii;
u64 rmii_pll1;
......@@ -156,8 +157,16 @@ struct sja1105_rule {
/* SJA1105_RULE_VL */
struct {
unsigned long destports;
enum sja1105_vl_type type;
unsigned long destports;
int sharindx;
int maxlen;
int ipv;
u64 base_time;
u64 cycle_time;
int num_entries;
struct action_gate_entry *entries;
struct flow_stats stats;
} vl;
};
};
......@@ -304,6 +313,8 @@ int sja1105_cls_flower_del(struct dsa_switch *ds, int port,
struct flow_cls_offload *cls, bool ingress);
int sja1105_cls_flower_add(struct dsa_switch *ds, int port,
struct flow_cls_offload *cls, bool ingress);
int sja1105_cls_flower_stats(struct dsa_switch *ds, int port,
struct flow_cls_offload *cls, bool ingress);
void sja1105_flower_setup(struct dsa_switch *ds);
void sja1105_flower_teardown(struct dsa_switch *ds);
struct sja1105_rule *sja1105_rule_find(struct sja1105_private *priv,
......
......@@ -309,7 +309,9 @@ int sja1105_cls_flower_add(struct dsa_switch *ds, int port,
struct sja1105_private *priv = ds->priv;
const struct flow_action_entry *act;
unsigned long cookie = cls->cookie;
bool routing_rule = false;
struct sja1105_key key;
bool gate_rule = false;
bool vl_rule = false;
int rc, i;
......@@ -332,6 +334,7 @@ int sja1105_cls_flower_add(struct dsa_switch *ds, int port,
case FLOW_ACTION_TRAP: {
int cpu = dsa_upstream_port(ds, port);
routing_rule = true;
vl_rule = true;
rc = sja1105_vl_redirect(priv, port, extack, cookie,
......@@ -350,6 +353,7 @@ int sja1105_cls_flower_add(struct dsa_switch *ds, int port,
return -EOPNOTSUPP;
}
routing_rule = true;
vl_rule = true;
rc = sja1105_vl_redirect(priv, port, extack, cookie,
......@@ -366,6 +370,21 @@ int sja1105_cls_flower_add(struct dsa_switch *ds, int port,
if (rc)
goto out;
break;
case FLOW_ACTION_GATE:
gate_rule = true;
vl_rule = true;
rc = sja1105_vl_gate(priv, port, extack, cookie,
&key, act->gate.index,
act->gate.prio,
act->gate.basetime,
act->gate.cycletime,
act->gate.cycletimeext,
act->gate.num_entries,
act->gate.entries);
if (rc)
goto out;
break;
default:
NL_SET_ERR_MSG_MOD(extack,
"Action not supported");
......@@ -374,8 +393,23 @@ int sja1105_cls_flower_add(struct dsa_switch *ds, int port,
}
}
if (vl_rule && !rc)
if (vl_rule && !rc) {
/* Delay scheduling configuration until DESTPORTS has been
* populated by all other actions.
*/
if (gate_rule) {
if (!routing_rule) {
NL_SET_ERR_MSG_MOD(extack,
"Can only offload gate action together with redirect or trap");
return -EOPNOTSUPP;
}
rc = sja1105_init_scheduling(priv);
if (rc)
goto out;
}
rc = sja1105_static_config_reload(priv, SJA1105_VIRTUAL_LINKS);
}
out:
return rc;
......@@ -421,6 +455,27 @@ int sja1105_cls_flower_del(struct dsa_switch *ds, int port,
return sja1105_static_config_reload(priv, SJA1105_BEST_EFFORT_POLICING);
}
int sja1105_cls_flower_stats(struct dsa_switch *ds, int port,
struct flow_cls_offload *cls, bool ingress)
{
struct sja1105_private *priv = ds->priv;
struct sja1105_rule *rule = sja1105_rule_find(priv, cls->cookie);
int rc;
if (!rule)
return 0;
if (rule->type != SJA1105_RULE_VL)
return 0;
rc = sja1105_vl_stats(priv, port, rule, &cls->stats,
cls->common.extack);
if (rc)
return rc;
return 0;
}
void sja1105_flower_setup(struct dsa_switch *ds)
{
struct sja1105_private *priv = ds->priv;
......
......@@ -2369,6 +2369,7 @@ static const struct dsa_switch_ops sja1105_switch_ops = {
.port_policer_del = sja1105_port_policer_del,
.cls_flower_add = sja1105_cls_flower_add,
.cls_flower_del = sja1105_cls_flower_del,
.cls_flower_stats = sja1105_cls_flower_stats,
};
static int sja1105_check_device_id(struct sja1105_private *priv)
......
......@@ -48,6 +48,19 @@ static inline s64 future_base_time(s64 base_time, s64 cycle_time, s64 now)
return base_time + n * cycle_time;
}
/* This is not a preprocessor macro because the "ns" argument may or may not be
* s64 at caller side. This ensures it is properly type-cast before div_s64.
*/
static inline s64 ns_to_sja1105_delta(s64 ns)
{
return div_s64(ns, 200);
}
static inline s64 sja1105_delta_to_ns(s64 delta)
{
return delta * 200;
}
struct sja1105_ptp_cmd {
u64 startptpcp; /* start toggling PTP_CLK pin */
u64 stopptpcp; /* stop toggling PTP_CLK pin */
......
......@@ -439,6 +439,7 @@ static struct sja1105_regs sja1105et_regs = {
.prod_id = 0x100BC3,
.status = 0x1,
.port_control = 0x11,
.vl_status = 0x10000,
.config = 0x020000,
.rgu = 0x100440,
/* UM10944.pdf, Table 86, ACU Register overview */
......@@ -472,6 +473,7 @@ static struct sja1105_regs sja1105pqrs_regs = {
.prod_id = 0x100BC3,
.status = 0x1,
.port_control = 0x12,
.vl_status = 0x10000,
.config = 0x020000,
.rgu = 0x100440,
/* UM10944.pdf, Table 86, ACU Register overview */
......
......@@ -302,6 +302,8 @@ struct sja1105_vl_lookup_entry {
u64 vlid;
};
};
/* Not part of hardware structure */
unsigned long flow_cookie;
};
struct sja1105_vl_policing_entry {
......
......@@ -7,7 +7,6 @@
#define SJA1105_TAS_CLKSRC_STANDALONE 1
#define SJA1105_TAS_CLKSRC_AS6802 2
#define SJA1105_TAS_CLKSRC_PTP 3
#define SJA1105_TAS_MAX_DELTA BIT(19)
#define SJA1105_GATE_MASK GENMASK_ULL(SJA1105_NUM_TC - 1, 0)
#define work_to_sja1105_tas(d) \
......@@ -15,22 +14,10 @@
#define tas_to_sja1105(d) \
container_of((d), struct sja1105_private, tas_data)
/* This is not a preprocessor macro because the "ns" argument may or may not be
* s64 at caller side. This ensures it is properly type-cast before div_s64.
*/
static s64 ns_to_sja1105_delta(s64 ns)
{
return div_s64(ns, 200);
}
static s64 sja1105_delta_to_ns(s64 delta)
{
return delta * 200;
}
static int sja1105_tas_set_runtime_params(struct sja1105_private *priv)
{
struct sja1105_tas_data *tas_data = &priv->tas_data;
struct sja1105_gating_config *gating_cfg = &tas_data->gating_cfg;
struct dsa_switch *ds = priv->ds;
s64 earliest_base_time = S64_MAX;
s64 latest_base_time = 0;
......@@ -59,6 +46,19 @@ static int sja1105_tas_set_runtime_params(struct sja1105_private *priv)
}
}
if (!list_empty(&gating_cfg->entries)) {
tas_data->enabled = true;
if (max_cycle_time < gating_cfg->cycle_time)
max_cycle_time = gating_cfg->cycle_time;
if (latest_base_time < gating_cfg->base_time)
latest_base_time = gating_cfg->base_time;
if (earliest_base_time > gating_cfg->base_time) {
earliest_base_time = gating_cfg->base_time;
its_cycle_time = gating_cfg->cycle_time;
}
}
if (!tas_data->enabled)
return 0;
......@@ -155,13 +155,14 @@ static int sja1105_tas_set_runtime_params(struct sja1105_private *priv)
* their "subschedule end index" (subscheind) equal to the last valid
* subschedule's end index (in this case 5).
*/
static int sja1105_init_scheduling(struct sja1105_private *priv)
int sja1105_init_scheduling(struct sja1105_private *priv)
{
struct sja1105_schedule_entry_points_entry *schedule_entry_points;
struct sja1105_schedule_entry_points_params_entry
*schedule_entry_points_params;
struct sja1105_schedule_params_entry *schedule_params;
struct sja1105_tas_data *tas_data = &priv->tas_data;
struct sja1105_gating_config *gating_cfg = &tas_data->gating_cfg;
struct sja1105_schedule_entry *schedule;
struct sja1105_table *table;
int schedule_start_idx;
......@@ -213,6 +214,11 @@ static int sja1105_init_scheduling(struct sja1105_private *priv)
}
}
if (!list_empty(&gating_cfg->entries)) {
num_entries += gating_cfg->num_entries;
num_cycles++;
}
/* Nothing to do */
if (!num_cycles)
return 0;
......@@ -312,6 +318,42 @@ static int sja1105_init_scheduling(struct sja1105_private *priv)
cycle++;
}
if (!list_empty(&gating_cfg->entries)) {
struct sja1105_gate_entry *e;
/* Relative base time */
s64 rbt;
schedule_start_idx = k;
schedule_end_idx = k + gating_cfg->num_entries - 1;
rbt = future_base_time(gating_cfg->base_time,
gating_cfg->cycle_time,
tas_data->earliest_base_time);
rbt -= tas_data->earliest_base_time;
entry_point_delta = ns_to_sja1105_delta(rbt) + 1;
schedule_entry_points[cycle].subschindx = cycle;
schedule_entry_points[cycle].delta = entry_point_delta;
schedule_entry_points[cycle].address = schedule_start_idx;
for (i = cycle; i < 8; i++)
schedule_params->subscheind[i] = schedule_end_idx;
list_for_each_entry(e, &gating_cfg->entries, list) {
schedule[k].delta = ns_to_sja1105_delta(e->interval);
schedule[k].destports = e->rule->vl.destports;
schedule[k].setvalid = true;
schedule[k].txen = true;
schedule[k].vlindex = e->rule->vl.sharindx;
schedule[k].winstindex = e->rule->vl.sharindx;
if (e->gate_state) /* Gate open */
schedule[k].winst = true;
else /* Gate closed */
schedule[k].winend = true;
k++;
}
}
return 0;
}
......@@ -415,6 +457,54 @@ sja1105_tas_check_conflicts(struct sja1105_private *priv, int port,
return false;
}
/* Check the tc-taprio configuration on @port for conflicts with the tc-gate
* global subschedule. If @port is -1, check it against all ports.
* To reuse the sja1105_tas_check_conflicts logic without refactoring it,
* convert the gating configuration to a dummy tc-taprio offload structure.
*/
bool sja1105_gating_check_conflicts(struct sja1105_private *priv, int port,
struct netlink_ext_ack *extack)
{
struct sja1105_gating_config *gating_cfg = &priv->tas_data.gating_cfg;
size_t num_entries = gating_cfg->num_entries;
struct tc_taprio_qopt_offload *dummy;
struct sja1105_gate_entry *e;
bool conflict;
int i = 0;
if (list_empty(&gating_cfg->entries))
return false;
dummy = kzalloc(sizeof(struct tc_taprio_sched_entry) * num_entries +
sizeof(struct tc_taprio_qopt_offload), GFP_KERNEL);
if (!dummy) {
NL_SET_ERR_MSG_MOD(extack, "Failed to allocate memory");
return true;
}
dummy->num_entries = num_entries;
dummy->base_time = gating_cfg->base_time;
dummy->cycle_time = gating_cfg->cycle_time;
list_for_each_entry(e, &gating_cfg->entries, list)
dummy->entries[i++].interval = e->interval;
if (port != -1) {
conflict = sja1105_tas_check_conflicts(priv, port, dummy);
} else {
for (port = 0; port < SJA1105_NUM_PORTS; port++) {
conflict = sja1105_tas_check_conflicts(priv, port,
dummy);
if (conflict)
break;
}
}
kfree(dummy);
return conflict;
}
int sja1105_setup_tc_taprio(struct dsa_switch *ds, int port,
struct tc_taprio_qopt_offload *admin)
{
......@@ -473,6 +563,11 @@ int sja1105_setup_tc_taprio(struct dsa_switch *ds, int port,
return -ERANGE;
}
if (sja1105_gating_check_conflicts(priv, port, NULL)) {
dev_err(ds->dev, "Conflict with tc-gate schedule\n");
return -ERANGE;
}
tas_data->offload[port] = taprio_offload_get(admin);
rc = sja1105_init_scheduling(priv);
......@@ -779,6 +874,8 @@ void sja1105_tas_setup(struct dsa_switch *ds)
INIT_WORK(&tas_data->tas_work, sja1105_tas_state_machine);
tas_data->state = SJA1105_TAS_STATE_DISABLED;
tas_data->last_op = SJA1105_PTP_NONE;
INIT_LIST_HEAD(&tas_data->gating_cfg.entries);
}
void sja1105_tas_teardown(struct dsa_switch *ds)
......
......@@ -6,6 +6,10 @@
#include <net/pkt_sched.h>
#define SJA1105_TAS_MAX_DELTA BIT(18)
struct sja1105_private;
#if IS_ENABLED(CONFIG_NET_DSA_SJA1105_TAS)
enum sja1105_tas_state {
......@@ -20,8 +24,23 @@ enum sja1105_ptp_op {
SJA1105_PTP_ADJUSTFREQ,
};
struct sja1105_gate_entry {
struct list_head list;
struct sja1105_rule *rule;
s64 interval;
u8 gate_state;
};
struct sja1105_gating_config {
u64 cycle_time;
s64 base_time;
int num_entries;
struct list_head entries;
};
struct sja1105_tas_data {
struct tc_taprio_qopt_offload *offload[SJA1105_NUM_PORTS];
struct sja1105_gating_config gating_cfg;
enum sja1105_tas_state state;
enum sja1105_ptp_op last_op;
struct work_struct tas_work;
......@@ -42,6 +61,11 @@ void sja1105_tas_clockstep(struct dsa_switch *ds);
void sja1105_tas_adjfreq(struct dsa_switch *ds);
bool sja1105_gating_check_conflicts(struct sja1105_private *priv, int port,
struct netlink_ext_ack *extack);
int sja1105_init_scheduling(struct sja1105_private *priv);
#else
/* C doesn't allow empty structures, bah! */
......@@ -63,6 +87,18 @@ static inline void sja1105_tas_clockstep(struct dsa_switch *ds) { }
static inline void sja1105_tas_adjfreq(struct dsa_switch *ds) { }
static inline bool
sja1105_gating_check_conflicts(struct dsa_switch *ds, int port,
struct netlink_ext_ack *extack)
{
return true;
}
static inline int sja1105_init_scheduling(struct sja1105_private *priv)
{
return 0;
}
#endif /* IS_ENABLED(CONFIG_NET_DSA_SJA1105_TAS) */
#endif /* _SJA1105_TAS_H */
This diff is collapsed.
......@@ -15,6 +15,16 @@ int sja1105_vl_delete(struct sja1105_private *priv, int port,
struct sja1105_rule *rule,
struct netlink_ext_ack *extack);
int sja1105_vl_gate(struct sja1105_private *priv, int port,
struct netlink_ext_ack *extack, unsigned long cookie,
struct sja1105_key *key, u32 index, s32 prio,
u64 base_time, u64 cycle_time, u64 cycle_time_ext,
u32 num_entries, struct action_gate_entry *entries);
int sja1105_vl_stats(struct sja1105_private *priv, int port,
struct sja1105_rule *rule, struct flow_stats *stats,
struct netlink_ext_ack *extack);
#else
static inline int sja1105_vl_redirect(struct sja1105_private *priv, int port,
......@@ -36,6 +46,27 @@ static inline int sja1105_vl_delete(struct sja1105_private *priv,
return -EOPNOTSUPP;
}
static inline int sja1105_vl_gate(struct sja1105_private *priv, int port,
struct netlink_ext_ack *extack,
unsigned long cookie,
struct sja1105_key *key, u32 index, s32 prio,
u64 base_time, u64 cycle_time,
u64 cycle_time_ext, u32 num_entries,
struct action_gate_entry *entries)
{
NL_SET_ERR_MSG_MOD(extack, "Virtual Links not compiled in");
return -EOPNOTSUPP;
}
static inline int sja1105_vl_stats(struct sja1105_private *priv, int port,
struct sja1105_rule *rule,
struct flow_stats *stats,
struct netlink_ext_ack *extack)
{
NL_SET_ERR_MSG_MOD(extack, "Virtual Links not compiled in");
return -EOPNOTSUPP;
}
#endif /* IS_ENABLED(CONFIG_NET_DSA_SJA1105_VL) */
#endif /* _SJA1105_VL_H */
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment