Commit fa062f2c authored by Steven Whitehouse's avatar Steven Whitehouse Committed by David S. Miller

[DECNET]: New autoconfiguration code for 2.5

parent 7fb54a70
......@@ -28,8 +28,36 @@ Don't turn on SIOCGIFCONF support for DECnet unless you are really sure
that you need it, in general you won't and it can cause ifconfig to
malfunction.
Run time configuration has changed slightly from the 2.4 system. If you
want to configure an endnode, then the simplified procedure is as follows:
o Set the MAC address on your ethernet card before starting _any_ other
network protocols.
As soon as your network card is brought into the UP state, DECnet should
start working. If you need something more complicated or are unsure how
to set the MAC address, see the next section. Also all configurations which
worked with 2.4 will work under 2.5 with no change.
3) Command line options
You can set a DECnet address on the kernel command line for compatibility
with the 2.4 configuration procedure, but in general its not needed any more.
If you do st a DECnet address on the command line, it has only one purpose
which is that its added to the addresses on the loopback device.
With 2.4 kernels, DECnet would only recognise addresses as local if they
were added to the loopback device. In 2.5, any local interface address
can be used to loop back to the local machine. Of course this does not
prevent you adding further addresses to the loopback device if you
want to.
N.B. Since the address list of an interface determines the addresses for
which "hello" messages are sent, if you don't set an address on the loopback
interface then you won't see any entries in /proc/net/neigh for the local
host until such time as you start a connection. This doesn't affect the
operation of the local communications in any other way though.
The kernel command line takes options looking like the following:
decnet=1,2
......@@ -51,7 +79,7 @@ parameters.
Currently the only supported devices are ethernet and ip_gre. The
ethernet address of your ethernet card has to be set according to the DECnet
address of the node in order for it to be recognised (and thus appear in
address of the node in order for it to be autoconfigured (and then appear in
/proc/net/decnet_dev). There is a utility available at the above
FTP sites called dn2ethaddr which can compute the correct ethernet
address to use. The address can be set by ifconfig either before at
......@@ -61,14 +89,22 @@ add the line:
MACADDR=AA:00:04:00:03:04
or something similar, to /etc/sysconfig/network-scripts/ifcfg-eth0 or
wherever your network card's configuration lives.
wherever your network card's configuration lives. Setting the MAC address
of your ethernet card to an address starting with "hi-ord" will cause a
DECnet address which matches to be added to the interface (which you can
verify with iproute2).
You will also need to set /proc/sys/net/decnet/default_device to the
The default device for routing can be set through the /proc filesystem
by setting /proc/sys/net/decnet/default_device to the
device you want DECnet to route packets out of when no specific route
is available. Usually this will be eth0, for example:
echo -n "eth0" >/proc/sys/net/decnet/default_device
If you don't set the default device, then it will default to the first
ethernet card which has been autoconfigured as described above. You can
confirm that by looking in the default_device file of course.
There is a list of what the other files under /proc/sys/net/decnet/ do
on the kernel patch web site (shown above).
......@@ -149,7 +185,36 @@ information (_most_ of which _is_ _essential_) includes:
You may also need to increase the length grabbed with the -s flag. The
-e flag also provides very useful information (ethernet MAC addresses))
7) Mailing list
7) MAC FAQ
A quick FAQ on ethernet MAC addresses to explain how Linux and DECnet
interact and how to get the best performance from your hardware.
Ethernet cards are designed to normally only pass received network frames
to a host computer when they are addressed to it, or to the broadcast address.
Linux has an interface which allows the setting of extra addresses for
an ethernet card to listen to. If the ethernet card supports it, the
filtering operation will be done in hardware, if not the extra unwanted packets
received will be discarded by the host computer. In the latter case,
significant processor time and bus bandwidth can be used up on a busy
network (see the NAPI documentation for a longer explanation of these
effects).
DECnet makes use of this interface to allow running DECnet on an ethernet
card which has already been configured using TCP/IP (presumably using the
built in MAC address of the card, as usual) and/or to allow multiple DECnet
addresses on each physical interface. If you do this, be aware that if your
ethernet card doesn't support perfect hashing in its MAC address filter
then your computer will be doing more work than required. Some cards
will simply set themselves into promiscuous mode in order to receive
packets from the DECnet specified addresses. So if you have one of these
cards its better to set the MAC address of the card as described above
to gain the best efficiency. Better still is to use a card which supports
NAPI as well.
8) Mailing list
If you are keen to get involved in development, or want to ask questions
about configuration, or even just report bugs, then there is a mailing
......@@ -157,7 +222,7 @@ list that you can join, details are at:
http://sourceforge.net/mail/?group_id=4993
8) Legal Info
9) Legal Info
The Linux DECnet project team have placed their code under the GPL. The
software is provided "as is" and without warranty express or implied.
......
......@@ -211,7 +211,6 @@ extern void dn_start_fast_timer(struct sock *sk);
extern void dn_stop_fast_timer(struct sock *sk);
extern dn_address decnet_address;
extern unsigned char decnet_ether_address[6];
extern int decnet_debug_level;
extern int decnet_time_wait;
extern int decnet_dn_count;
......
......@@ -57,7 +57,7 @@ struct dn_ifaddr {
* up() - Called to initialize device, return value can veto use of
* device with DECnet.
* down() - Called to turn device off when it goes down
* timer3() - Called when timer 3 goes off
* timer3() - Called once for each ifaddr when timer 3 goes off
*
* sysctl - Hook for sysctl things
*
......@@ -78,7 +78,7 @@ struct dn_dev_parms {
int ctl_name; /* Index for sysctl */
int (*up)(struct net_device *);
void (*down)(struct net_device *);
void (*timer3)(struct net_device *);
void (*timer3)(struct net_device *, struct dn_ifaddr *ifa);
void *sysctl;
};
......@@ -167,7 +167,9 @@ extern void dn_dev_hello(struct sk_buff *skb);
extern void dn_dev_up(struct net_device *);
extern void dn_dev_down(struct net_device *);
extern struct net_device *decnet_default_device;
extern int dn_dev_set_default(struct net_device *dev, int force);
extern struct net_device *dn_dev_get_default(void);
extern int dn_dev_bind_default(dn_address *addr);
static __inline__ int dn_dev_islocal(struct net_device *dev, dn_address addr)
{
......
......@@ -96,7 +96,7 @@ static inline void dn_rt_send(struct sk_buff *skb)
dev_queue_xmit(skb);
}
static inline void dn_rt_finish_output(struct sk_buff *skb, char *dst)
static inline void dn_rt_finish_output(struct sk_buff *skb, char *dst, char *src)
{
struct net_device *dev = skb->dev;
......@@ -104,7 +104,7 @@ static inline void dn_rt_finish_output(struct sk_buff *skb, char *dst)
dst = NULL;
if (!dev->hard_header || (dev->hard_header(skb, dev, ETH_P_DNA_RT,
dst, NULL, skb->len) >= 0))
dst, src, skb->len) >= 0))
dn_rt_send(skb);
else
kfree_skb(skb);
......
......@@ -35,14 +35,11 @@ Steve's quick list of things that need finishing off:
file)
o Find all the commonality between DECnet and IPv4 routing code and extract
it into a small library of routines. [probably a project for 2.5.xx]
it into a small library of routines. [probably a project for 2.7.xx]
o Test ip_gre tunneling works... it did the last time I tested it and it
will have to if I'm to test routing properly.
o Hello messages should be generated for each primary address on each
interface.
o Add the routing message grabbing netfilter module [written, tested,
awaiting merge]
......@@ -55,3 +52,5 @@ Steve's quick list of things that need finishing off:
o DECnet sendpages() function
o AIO for DECnet
......@@ -100,6 +100,7 @@ Version 0.0.6 2.1.110 07-aug-98 Eduardo Marcelo Serrat
#include <linux/module.h>
#include <linux/errno.h>
#include <linux/types.h>
#include <linux/slab.h>
#include <linux/socket.h>
#include <linux/in.h>
#include <linux/kernel.h>
......@@ -131,21 +132,20 @@ Version 0.0.6 2.1.110 07-aug-98 Eduardo Marcelo Serrat
#include <net/dn_fib.h>
#include <net/dn_neigh.h>
static void dn_keepalive(struct sock *sk);
struct dn_sock {
struct sock sk;
struct dn_scp scp;
};
/*
* decnet_address is kept in network order, decnet_ether_address is kept
* as a string of bytes.
*/
dn_address decnet_address = 0;
unsigned char decnet_ether_address[ETH_ALEN] = { 0xAA, 0x00, 0x04, 0x00, 0x00, 0x00 };
static void dn_keepalive(struct sock *sk);
#define DN_SK_HASH_SHIFT 8
#define DN_SK_HASH_SIZE (1 << DN_SK_HASH_SHIFT)
#define DN_SK_HASH_MASK (DN_SK_HASH_SIZE - 1)
static kmem_cache_t *dn_sk_cachep;
static struct proto_ops dn_proto_ops;
rwlock_t dn_hash_lock = RW_LOCK_UNLOCKED;
static rwlock_t dn_hash_lock = RW_LOCK_UNLOCKED;
static struct sock *dn_sk_hash[DN_SK_HASH_SIZE];
static struct sock *dn_wild_sk;
......@@ -473,17 +473,16 @@ struct sock *dn_alloc_sock(struct socket *sock, int gfp)
struct sock *sk;
struct dn_scp *scp;
if ((sk = sk_alloc(PF_DECnet, gfp, 1, NULL)) == NULL)
if ((sk = sk_alloc(PF_DECnet, gfp, sizeof(struct dn_sock), dn_sk_cachep)) == NULL)
goto no_sock;
scp = kmalloc(sizeof(*scp), gfp);
if (!scp)
goto free_sock;
scp = (struct dn_scp *)(sk + 1);
DN_SK(sk) = scp;
if (sock) {
sock->ops = &dn_proto_ops;
}
sock_init_data(sock,sk);
DN_SK(sk) = scp;
sk->backlog_rcv = dn_nsp_backlog_rcv;
sk->destruct = dn_destruct;
......@@ -544,8 +543,7 @@ struct sock *dn_alloc_sock(struct socket *sock, int gfp)
MOD_INC_USE_COUNT;
return sk;
free_sock:
sk_free(sk);
no_sock:
return NULL;
}
......@@ -771,9 +769,6 @@ static int dn_bind(struct socket *sock, struct sockaddr *uaddr, int addr_len)
struct net_device *dev;
int rv;
if (sk->zapped == 0)
return -EINVAL;
if (addr_len != sizeof(struct sockaddr_dn))
return -EINVAL;
......@@ -783,19 +778,30 @@ static int dn_bind(struct socket *sock, struct sockaddr *uaddr, int addr_len)
if (dn_ntohs(saddr->sdn_nodeaddrl) && (dn_ntohs(saddr->sdn_nodeaddrl) != 2))
return -EINVAL;
if (saddr->sdn_objnum && !capable(CAP_NET_BIND_SERVICE))
return -EPERM;
if (dn_ntohs(saddr->sdn_objnamel) > DN_MAXOBJL)
return -EINVAL;
if (saddr->sdn_flags & ~SDF_WILD)
return -EINVAL;
if (saddr->sdn_flags & SDF_WILD) {
if (!capable(CAP_NET_BIND_SERVICE))
return -EPERM;
} else {
#if 1
if (!capable(CAP_NET_BIND_SERVICE) && saddr->sdn_objnum ||
(saddr->sdn_flags & SDF_WILD))
return -EACCES;
#else
/*
* Maybe put the default actions in the default security ops for
* dn_prot_sock ? Would be nice if the capable call would go there
* too.
*/
if (security_ops->dn_prot_sock(saddr) &&
!capable(CAP_NET_BIND_SERVICE) ||
saddr->sdn_objnum || (saddr->sdn_flags & SDF_WILD))
return -EACCES;
#endif
if (!(saddr->sdn_flags & SDF_WILD)) {
if (dn_ntohs(saddr->sdn_nodeaddrl)) {
read_lock(&dev_base_lock);
for(dev = dev_base; dev; dev = dev->next) {
......@@ -810,12 +816,18 @@ static int dn_bind(struct socket *sock, struct sockaddr *uaddr, int addr_len)
}
}
rv = -EINVAL;
lock_sock(sk);
if (sk->zapped != 0) {
memcpy(&scp->addr, saddr, addr_len);
sk->zapped = 0;
memcpy(&scp->addr, saddr, addr_len);
sk->zapped = 0;
if ((rv = dn_hash_sock(sk)) != 0)
sk->zapped = 1;
rv = dn_hash_sock(sk);
if (rv) {
sk->zapped = 1;
}
}
release_sock(sk);
return rv;
}
......@@ -825,6 +837,7 @@ static int dn_auto_bind(struct socket *sock)
{
struct sock *sk = sock->sk;
struct dn_scp *scp = DN_SK(sk);
int rv;
sk->zapped = 0;
......@@ -844,13 +857,18 @@ static int dn_auto_bind(struct socket *sock)
scp->accessdata.acc_accl = 0;
memset(scp->accessdata.acc_acc, 0, 40);
}
/* End of compatibility stuff */
scp->addr.sdn_add.a_len = dn_htons(2);
*(dn_address *)scp->addr.sdn_add.a_addr = decnet_address;
dn_hash_sock(sk);
rv = dn_dev_bind_default((dn_address *)scp->addr.sdn_add.a_addr);
if (rv == 0) {
rv = dn_hash_sock(sk);
if (rv) {
sk->zapped = 1;
}
}
return 0;
return rv;
}
......@@ -1209,6 +1227,7 @@ static int dn_ioctl(struct socket *sock, unsigned int cmd, unsigned long arg)
return dn_fib_ioctl(sock, cmd, arg);
#endif /* CONFIG_DECNET_ROUTER */
#if 0
case OSIOCSNETADDR:
if (!capable(CAP_NET_ADMIN)) {
err = -EPERM;
......@@ -1218,7 +1237,6 @@ static int dn_ioctl(struct socket *sock, unsigned int cmd, unsigned long arg)
dn_dev_devices_off();
decnet_address = (unsigned short)arg;
dn_dn2eth(decnet_ether_address, dn_ntohs(decnet_address));
dn_dev_devices_on();
err = 0;
......@@ -1227,6 +1245,7 @@ static int dn_ioctl(struct socket *sock, unsigned int cmd, unsigned long arg)
case OSIOCGNETADDR:
err = put_user(decnet_address, (unsigned short *)arg);
break;
#endif
case SIOCGIFCONF:
case SIOCGIFFLAGS:
case SIOCGIFBRDADDR:
......@@ -2227,38 +2246,24 @@ void dn_unregister_sysctl(void);
#endif
#ifdef MODULE
MODULE_DESCRIPTION("The Linux DECnet Network Protocol");
MODULE_AUTHOR("Linux DECnet Project Team");
MODULE_LICENSE("GPL");
static int addr[2] = {0, 0};
MODULE_PARM(addr, "2i");
MODULE_PARM_DESC(addr, "The DECnet address of this machine: area,node");
#endif
static char banner[] __initdata = KERN_INFO "NET4: DECnet for Linux: V.2.4.20-pre1s (C) 1995-2002 Linux DECnet Project Team\n";
static char banner[] __initdata = KERN_INFO "NET4: DECnet for Linux: V.2.5.40s (C) 1995-2002 Linux DECnet Project Team\n";
static int __init decnet_init(void)
{
#ifdef MODULE
if (addr[0] > 63 || addr[0] < 0) {
printk(KERN_ERR "DECnet: Area must be between 0 and 63");
return 1;
}
if (addr[1] > 1023 || addr[1] < 0) {
printk(KERN_ERR "DECnet: Node must be between 0 and 1023");
return 1;
}
decnet_address = dn_htons((addr[0] << 10) | addr[1]);
dn_dn2eth(decnet_ether_address, dn_ntohs(decnet_address));
#endif
printk(banner);
dn_sk_cachep = kmem_cache_create("decnet_socket_cache",
sizeof(struct dn_sock),
0, SLAB_HWCACHE_ALIGN,
NULL, NULL);
if (!dn_sk_cachep)
return -ENOMEM;
sock_register(&dn_family_ops);
dev_add_pack(&dn_dix_packet_type);
register_netdevice_notifier(&dn_dev_notifier);
......@@ -2288,21 +2293,6 @@ static int __init decnet_init(void)
}
#ifndef MODULE
static int __init decnet_setup(char *str)
{
unsigned short area = simple_strtoul(str, &str, 0);
unsigned short node = simple_strtoul(*str > 0 ? ++str : str, &str, 0);
decnet_address = dn_htons(area << 10 | node);
dn_dn2eth(decnet_ether_address, dn_ntohs(decnet_address));
return 1;
}
__setup("decnet=", decnet_setup);
#endif
static void __exit decnet_exit(void)
{
sock_unregister(AF_DECnet);
......@@ -2323,6 +2313,8 @@ static void __exit decnet_exit(void)
#endif /* CONFIG_DECNET_ROUTER */
proc_net_remove("decnet");
kmem_cache_destroy(dn_sk_cachep);
}
module_init(decnet_init);
......
......@@ -23,6 +23,8 @@
*/
#include <linux/config.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/net.h>
#include <linux/netdevice.h>
#include <linux/proc_fs.h>
......@@ -52,16 +54,23 @@ static unsigned char dn_eco_version[3] = {0x02,0x00,0x00};
extern struct neigh_table dn_neigh_table;
struct net_device *decnet_default_device;
/*
* decnet_address is kept in network order.
*/
dn_address decnet_address = 0;
static rwlock_t dndev_lock = RW_LOCK_UNLOCKED;
static struct net_device *decnet_default_device;
static struct dn_dev *dn_dev_create(struct net_device *dev, int *err);
static void dn_dev_delete(struct net_device *dev);
static void rtmsg_ifa(int event, struct dn_ifaddr *ifa);
static int dn_eth_up(struct net_device *);
static void dn_send_brd_hello(struct net_device *dev);
static void dn_eth_down(struct net_device *);
static void dn_send_brd_hello(struct net_device *dev, struct dn_ifaddr *ifa);
#if 0
static void dn_send_ptp_hello(struct net_device *dev);
static void dn_send_ptp_hello(struct net_device *dev, struct dn_ifaddr *ifa);
#endif
static struct dn_dev_parms dn_dev_list[] = {
......@@ -75,6 +84,7 @@ static struct dn_dev_parms dn_dev_list[] = {
.name = "ethernet",
.ctl_name = NET_DECNET_CONF_ETHER,
.up = dn_eth_up,
.down = dn_eth_down,
.timer3 = dn_send_brd_hello,
},
{
......@@ -249,6 +259,51 @@ static void dn_dev_sysctl_unregister(struct dn_dev_parms *parms)
}
}
struct net_device *dn_dev_get_default(void)
{
struct net_device *dev;
read_lock(&dndev_lock);
dev = decnet_default_device;
if (dev) {
if (dev->dn_ptr)
dev_hold(dev);
else
dev = NULL;
}
read_unlock(&dndev_lock);
return dev;
}
int dn_dev_set_default(struct net_device *dev, int force)
{
struct net_device *old = NULL;
int rv = -EBUSY;
if (!dev->dn_ptr)
return -ENODEV;
write_lock(&dndev_lock);
if (force || decnet_default_device == NULL) {
old = decnet_default_device;
decnet_default_device = dev;
rv = 0;
}
write_unlock(&dndev_lock);
if (old)
dev_put(dev);
return rv;
}
static void dn_dev_check_default(struct net_device *dev)
{
write_lock(&dndev_lock);
if (dev == decnet_default_device) {
decnet_default_device = NULL;
} else {
dev = NULL;
}
write_unlock(&dndev_lock);
if (dev)
dev_put(dev);
}
static int dn_forwarding_proc(ctl_table *table, int write,
struct file *filep,
......@@ -364,9 +419,20 @@ static __inline__ void dn_dev_free_ifa(struct dn_ifaddr *ifa)
static void dn_dev_del_ifa(struct dn_dev *dn_db, struct dn_ifaddr **ifap, int destroy)
{
struct dn_ifaddr *ifa1 = *ifap;
unsigned char mac_addr[6];
struct net_device *dev = dn_db->dev;
ASSERT_RTNL();
*ifap = ifa1->ifa_next;
if (dn_db->dev->type == ARPHRD_ETHER) {
if (ifa1->ifa_local != dn_htons(dn_eth2dn(dev->dev_addr))) {
dn_dn2eth(mac_addr, ifa1->ifa_local);
dev_mc_delete(dev, mac_addr, ETH_ALEN, 0);
}
}
rtmsg_ifa(RTM_DELADDR, ifa1);
if (destroy) {
......@@ -379,9 +445,25 @@ static void dn_dev_del_ifa(struct dn_dev *dn_db, struct dn_ifaddr **ifap, int de
static int dn_dev_insert_ifa(struct dn_dev *dn_db, struct dn_ifaddr *ifa)
{
/*
* FIXME: Duplicate check here.
*/
struct net_device *dev = dn_db->dev;
struct dn_ifaddr *ifa1;
unsigned char mac_addr[6];
ASSERT_RTNL();
/* Check for duplicates */
for(ifa1 = dn_db->ifa_list; ifa1; ifa1 = ifa1->ifa_next) {
if (ifa1->ifa_local == ifa->ifa_local)
return -EEXIST;
}
if (dev->type == ARPHRD_ETHER) {
if (ifa->ifa_local != dn_htons(dn_eth2dn(dev->dev_addr))) {
dn_dn2eth(mac_addr, ifa->ifa_local);
dev_mc_add(dev, mac_addr, ETH_ALEN, 0);
dev_mc_upload(dev);
}
}
ifa->ifa_next = dn_db->ifa_list;
dn_db->ifa_list = ifa;
......@@ -394,6 +476,7 @@ static int dn_dev_insert_ifa(struct dn_dev *dn_db, struct dn_ifaddr *ifa)
static int dn_dev_set_ifa(struct net_device *dev, struct dn_ifaddr *ifa)
{
struct dn_dev *dn_db = dev->dn_ptr;
int rv;
if (dn_db == NULL) {
int err;
......@@ -407,7 +490,10 @@ static int dn_dev_set_ifa(struct net_device *dev, struct dn_ifaddr *ifa)
if (dev->flags & IFF_LOOPBACK)
ifa->ifa_scope = RT_SCOPE_HOST;
return dn_dev_insert_ifa(dn_db, ifa);
rv = dn_dev_insert_ifa(dn_db, ifa);
if (rv)
dn_dev_free_ifa(ifa);
return rv;
}
......@@ -538,6 +624,7 @@ static int dn_dev_rtm_newaddr(struct sk_buff *skb, struct nlmsghdr *nlh, void *a
struct dn_dev *dn_db;
struct ifaddrmsg *ifm = NLMSG_DATA(nlh);
struct dn_ifaddr *ifa;
int rv;
if (rta[IFA_LOCAL-1] == NULL)
return -EINVAL;
......@@ -564,7 +651,10 @@ static int dn_dev_rtm_newaddr(struct sk_buff *skb, struct nlmsghdr *nlh, void *a
else
memcpy(ifa->ifa_label, dev->name, IFNAMSIZ);
return dn_dev_insert_ifa(dn_db, ifa);
rv = dn_dev_insert_ifa(dn_db, ifa);
if (rv)
dn_dev_free_ifa(ifa);
return rv;
}
static int dn_dev_fill_ifaddr(struct sk_buff *skb, struct dn_ifaddr *ifa,
......@@ -651,7 +741,52 @@ static int dn_dev_dump_ifaddr(struct sk_buff *skb, struct netlink_callback *cb)
return skb->len;
}
static void dn_send_endnode_hello(struct net_device *dev)
static int dn_dev_get_first(struct net_device *dev, dn_address *addr)
{
struct dn_dev *dn_db = (struct dn_dev *)dev->dn_ptr;
struct dn_ifaddr *ifa;
int rv = -ENODEV;
if (dn_db == NULL)
goto out;
ifa = dn_db->ifa_list;
if (ifa != NULL) {
*addr = ifa->ifa_local;
rv = 0;
}
out:
return rv;
}
/*
* Find a default address to bind to.
*
* This is one of those areas where the initial VMS concepts don't really
* map onto the Linux concepts, and since we introduced multiple addresses
* per interface we have to cope with slightly odd ways of finding out what
* "our address" really is. Mostly its not a problem; for this we just guess
* a sensible default. Eventually the routing code will take care of all the
* nasties for us I hope.
*/
int dn_dev_bind_default(dn_address *addr)
{
struct net_device *dev;
int rv;
dev = dn_dev_get_default();
last_chance:
if (dev) {
read_lock(&dev_base_lock);
rv = dn_dev_get_first(dev, addr);
read_unlock(&dev_base_lock);
dev_put(dev);
if (rv == 0 || dev == &loopback_dev)
return rv;
}
dev = &loopback_dev;
dev_hold(dev);
goto last_chance;
}
static void dn_send_endnode_hello(struct net_device *dev, struct dn_ifaddr *ifa)
{
struct endnode_hello_message *msg;
struct sk_buff *skb = NULL;
......@@ -667,7 +802,7 @@ static void dn_send_endnode_hello(struct net_device *dev)
msg->msgflg = 0x0D;
memcpy(msg->tiver, dn_eco_version, 3);
memcpy(msg->id, decnet_ether_address, 6);
dn_dn2eth(msg->id, ifa->ifa_local);
msg->iinfo = DN_RT_INFO_ENDN;
msg->blksize = dn_htons(dn_db->parms.blksize);
msg->area = 0x00;
......@@ -689,7 +824,7 @@ static void dn_send_endnode_hello(struct net_device *dev)
skb->nh.raw = skb->data;
dn_rt_finish_output(skb, dn_rt_all_rt_mcast);
dn_rt_finish_output(skb, dn_rt_all_rt_mcast, msg->id);
}
......@@ -697,7 +832,7 @@ static void dn_send_endnode_hello(struct net_device *dev)
#define DRDELAY (5 * HZ)
static int dn_am_i_a_router(struct dn_neigh *dn, struct dn_dev *dn_db)
static int dn_am_i_a_router(struct dn_neigh *dn, struct dn_dev *dn_db, struct dn_ifaddr *ifa)
{
/* First check time since device went up */
if ((jiffies - dn_db->uptime) < DRDELAY)
......@@ -715,13 +850,13 @@ static int dn_am_i_a_router(struct dn_neigh *dn, struct dn_dev *dn_db)
if (dn->priority != dn_db->parms.priority)
return 0;
if (dn_ntohs(dn->addr) < dn_ntohs(decnet_address))
if (dn_ntohs(dn->addr) < dn_ntohs(ifa->ifa_local))
return 1;
return 0;
}
static void dn_send_router_hello(struct net_device *dev)
static void dn_send_router_hello(struct net_device *dev, struct dn_ifaddr *ifa)
{
int n;
struct dn_dev *dn_db = dev->dn_ptr;
......@@ -731,6 +866,7 @@ static void dn_send_router_hello(struct net_device *dev)
unsigned char *ptr;
unsigned char *i1, *i2;
unsigned short *pktlen;
char *src;
if (dn_db->parms.blksize < (26 + 7))
return;
......@@ -753,7 +889,8 @@ static void dn_send_router_hello(struct net_device *dev)
*ptr++ = 2; /* ECO */
*ptr++ = 0;
*ptr++ = 0;
memcpy(ptr, decnet_ether_address, ETH_ALEN);
dn_dn2eth(ptr, ifa->ifa_local);
src = ptr;
ptr += ETH_ALEN;
*ptr++ = dn_db->parms.forwarding == 1 ?
DN_RT_INFO_L1RT : DN_RT_INFO_L2RT;
......@@ -781,34 +918,34 @@ static void dn_send_router_hello(struct net_device *dev)
skb->nh.raw = skb->data;
if (dn_am_i_a_router(dn, dn_db)) {
if (dn_am_i_a_router(dn, dn_db, ifa)) {
struct sk_buff *skb2 = skb_copy(skb, GFP_ATOMIC);
if (skb2) {
dn_rt_finish_output(skb2, dn_rt_all_end_mcast);
dn_rt_finish_output(skb2, dn_rt_all_end_mcast, src);
}
}
dn_rt_finish_output(skb, dn_rt_all_rt_mcast);
dn_rt_finish_output(skb, dn_rt_all_rt_mcast, src);
}
static void dn_send_brd_hello(struct net_device *dev)
static void dn_send_brd_hello(struct net_device *dev, struct dn_ifaddr *ifa)
{
struct dn_dev *dn_db = (struct dn_dev *)dev->dn_ptr;
if (dn_db->parms.forwarding == 0)
dn_send_endnode_hello(dev);
dn_send_endnode_hello(dev, ifa);
else
dn_send_router_hello(dev);
dn_send_router_hello(dev, ifa);
}
#else
static void dn_send_brd_hello(struct net_device *dev)
static void dn_send_brd_hello(struct net_device *dev, struct dn_ifaddr *ifa)
{
dn_send_endnode_hello(dev);
dn_send_endnode_hello(dev, ifa);
}
#endif
#if 0
static void dn_send_ptp_hello(struct net_device *dev)
static void dn_send_ptp_hello(struct net_device *dev, struct dn_ifaddr *ifa)
{
int tdlen = 16;
int size = dev->hard_header_len + 2 + 4 + tdlen;
......@@ -817,6 +954,7 @@ static void dn_send_ptp_hello(struct net_device *dev)
int i;
unsigned char *ptr;
struct dn_neigh *dn = (struct dn_neigh *)dn_db->router;
char src[ETH_ALEN];
if (skb == NULL)
return ;
......@@ -826,21 +964,15 @@ static void dn_send_ptp_hello(struct net_device *dev)
ptr = skb_put(skb, 2 + 4 + tdlen);
*ptr++ = DN_RT_PKT_HELO;
*((dn_address *)ptr) = decnet_address;
*((dn_address *)ptr) = ifa->ifa_local;
ptr += 2;
*ptr++ = tdlen;
for(i = 0; i < tdlen; i++)
*ptr++ = 0252;
if (dn_am_i_a_router(dn, dn_db)) {
struct sk_buff *skb2 = skb_copy(skb, GFP_ATOMIC);
if (skb2) {
dn_rt_finish_output(skb2, dn_rt_all_end_mcast);
}
}
dn_rt_finish_output(skb, dn_rt_all_rt_mcast);
dn_dn2eth(src, ifa->ifa_local);
dn_rt_finish_output(skb, dn_rt_all_rt_mcast, src);
}
#endif
......@@ -860,16 +992,31 @@ static int dn_eth_up(struct net_device *dev)
return 0;
}
static void dn_eth_down(struct net_device *dev)
{
struct dn_dev *dn_db = dev->dn_ptr;
if (dn_db->parms.forwarding == 0)
dev_mc_delete(dev, dn_rt_all_end_mcast, ETH_ALEN, 0);
else
dev_mc_delete(dev, dn_rt_all_rt_mcast, ETH_ALEN, 0);
}
static void dn_dev_set_timer(struct net_device *dev);
static void dn_dev_timer_func(unsigned long arg)
{
struct net_device *dev = (struct net_device *)arg;
struct dn_dev *dn_db = dev->dn_ptr;
struct dn_ifaddr *ifa;
if (dn_db->t3 <= dn_db->parms.t2) {
if (dn_db->parms.timer3)
dn_db->parms.timer3(dev);
if (dn_db->parms.timer3) {
for(ifa = dn_db->ifa_list; ifa; ifa = ifa->ifa_next) {
if (!(ifa->ifa_flags & IFA_F_SECONDARY))
dn_db->parms.timer3(dev, ifa);
}
}
dn_db->t3 = dn_db->parms.t3;
} else {
dn_db->t3 -= dn_db->parms.t2;
......@@ -917,8 +1064,6 @@ struct dn_dev *dn_dev_create(struct net_device *dev, int *err)
dn_db->dev = dev;
init_timer(&dn_db->timer);
memcpy(dn_db->addr, decnet_ether_address, ETH_ALEN); /* To go... */
dn_db->uptime = jiffies;
if (dn_db->parms.up) {
if (dn_db->parms.up(dev) < 0) {
......@@ -929,7 +1074,6 @@ struct dn_dev *dn_dev_create(struct net_device *dev, int *err)
}
dn_db->neigh_parms = neigh_parms_alloc(dev, &dn_neigh_table);
/* dn_db->neigh_parms->neigh_setup = dn_db->parms.neigh_setup; */
dn_dev_sysctl_register(dev, &dn_db->parms);
......@@ -945,27 +1089,64 @@ struct dn_dev *dn_dev_create(struct net_device *dev, int *err)
* the loopback device & ethernet devices with correct
* MAC addreses automatically. Others must be started
* specifically.
*
* FIXME: How should we configure the loopback address ? If we could dispense
* with using decnet_address here and for autobind, it will be one less thing
* for users to worry about setting up.
*/
void dn_dev_up(struct net_device *dev)
{
struct dn_ifaddr *ifa;
dn_address addr = decnet_address;
int maybe_default = 0;
struct dn_dev *dn_db = (struct dn_dev *)dev->dn_ptr;
if ((dev->type != ARPHRD_ETHER) && (dev->type != ARPHRD_LOOPBACK))
return;
if (dev->type == ARPHRD_ETHER)
if (memcmp(dev->dev_addr, decnet_ether_address, ETH_ALEN) != 0)
/*
* Need to ensure that loopback device has a dn_db attached to it
* to allow creation of neighbours against it, even though it might
* not have a local address of its own. Might as well do the same for
* all autoconfigured interfaces.
*/
if (dn_db == NULL) {
int err;
dn_db = dn_dev_create(dev, &err);
if (dn_db == NULL)
return;
}
if (dev->type == ARPHRD_ETHER) {
if (memcmp(dev->dev_addr, dn_hiord, 4) != 0)
return;
addr = dn_htons(dn_eth2dn(dev->dev_addr));
maybe_default = 1;
}
if (addr == 0)
return;
if ((ifa = dn_dev_alloc_ifa()) == NULL)
return;
ifa->ifa_local = decnet_address;
ifa->ifa_local = addr;
ifa->ifa_flags = 0;
ifa->ifa_scope = RT_SCOPE_UNIVERSE;
strcpy(ifa->ifa_label, dev->name);
dn_dev_set_ifa(dev, ifa);
/*
* Automagically set the default device to the first automatically
* configured ethernet card in the system.
*/
if (maybe_default) {
dev_hold(dev);
if (dn_dev_set_default(dev, 0))
dev_put(dev);
}
}
static void dn_dev_delete(struct net_device *dev)
......@@ -976,14 +1157,10 @@ static void dn_dev_delete(struct net_device *dev)
return;
del_timer_sync(&dn_db->timer);
dn_dev_sysctl_unregister(&dn_db->parms);
dn_dev_check_default(dev);
neigh_ifdown(&dn_neigh_table, dev);
if (dev == decnet_default_device)
decnet_default_device = NULL;
if (dn_db->parms.down)
dn_db->parms.down(dev);
......@@ -1204,8 +1381,28 @@ static struct rtnetlink_link dnet_rtnetlink_table[RTM_MAX-RTM_BASE+1] =
#endif
};
#ifdef MODULE
static int addr[2] = {0, 0};
MODULE_PARM(addr, "2i");
MODULE_PARM_DESC(addr, "The DECnet address of this machine: area,node");
#endif
void __init dn_dev_init(void)
{
#ifdef MODULE
if (addr[0] > 63 || addr[0] < 0) {
printk(KERN_ERR "DECnet: Area must be between 0 and 63");
return 1;
}
if (addr[1] > 1023 || addr[1] < 0) {
printk(KERN_ERR "DECnet: Node must be between 0 and 1023");
return 1;
}
decnet_address = dn_htons((addr[0] << 10) | addr[1]);
#endif
dn_dev_devices_on();
#ifdef CONFIG_DECNET_SIOCGIFCONF
......@@ -1247,3 +1444,18 @@ void __exit dn_dev_cleanup(void)
dn_dev_devices_off();
}
#ifndef MODULE
static int __init decnet_setup(char *str)
{
unsigned short area = simple_strtoul(str, &str, 0);
unsigned short node = simple_strtoul(*str > 0 ? ++str : str, &str, 0);
decnet_address = dn_htons(area << 10 | node);
return 1;
}
__setup("decnet=", decnet_setup);
#endif
......@@ -181,10 +181,13 @@ static void dn_short_error_report(struct neighbour *neigh, struct sk_buff *skb)
static int dn_neigh_output_packet(struct sk_buff *skb)
{
struct dst_entry *dst = skb->dst;
struct dn_route *rt = (struct dn_route *)dst;
struct neighbour *neigh = dst->neighbour;
struct net_device *dev = neigh->dev;
char mac_addr[ETH_ALEN];
if (!dev->hard_header || dev->hard_header(skb, dev, ntohs(skb->protocol), neigh->ha, NULL, skb->len) >= 0)
dn_dn2eth(mac_addr, rt->rt_saddr);
if (!dev->hard_header || dev->hard_header(skb, dev, ntohs(skb->protocol), neigh->ha, mac_addr, skb->len) >= 0)
return neigh->ops->queue_xmit(skb);
if (net_ratelimit())
......
......@@ -334,7 +334,7 @@ static int dn_return_short(struct sk_buff *skb)
*dst = tmp;
skb->pkt_type = PACKET_OUTGOING;
dn_rt_finish_output(skb, NULL);
dn_rt_finish_output(skb, NULL, NULL);
return NET_RX_SUCCESS;
}
......@@ -380,7 +380,7 @@ static int dn_return_long(struct sk_buff *skb)
memcpy(dst_addr, tmp, ETH_ALEN);
skb->pkt_type = PACKET_OUTGOING;
dn_rt_finish_output(skb, tmp);
dn_rt_finish_output(skb, dst_addr, src_addr);
return NET_RX_SUCCESS;
}
......@@ -641,7 +641,9 @@ static int dn_forward(struct sk_buff *skb)
struct dn_skb_cb *cb = DN_SKB_CB(skb);
struct dst_entry *dst = skb->dst;
struct neighbour *neigh;
#ifdef CONFIG_NETFILTER
struct net_device *dev = skb->dev;
#endif
int err = -EINVAL;
if ((neigh = dst->neighbour) == NULL)
......@@ -711,10 +713,11 @@ static int dn_rt_bug(struct sk_buff *skb)
static int dn_route_output_slow(struct dst_entry **pprt, dn_address dst, dn_address src, int flags)
{
struct dn_route *rt = NULL;
struct net_device *dev = decnet_default_device;
struct net_device *dev = NULL;
struct neighbour *neigh = NULL;
struct dn_dev *dn_db;
unsigned hash;
#ifdef CONFIG_DECNET_ROUTER
struct dn_fib_key key;
struct dn_fib_res res;
......@@ -765,13 +768,25 @@ static int dn_route_output_slow(struct dst_entry **pprt, dn_address dst, dn_addr
goto got_route;
}
dev = dn_dev_get_default();
if (dev == NULL)
return -EINVAL;
dn_db = dev->dn_ptr;
if (dn_db == NULL)
/* Check to see if its one of our own local addresses */
if (dn_dev_islocal(dev, dst)) {
struct net_device *lo = &loopback_dev;
if (lo->dn_ptr) {
neigh = __neigh_lookup(&dn_neigh_table, &dst, lo, 1);
if (neigh)
goto got_route;
}
if (net_ratelimit())
printk("dn_route_output_slow: Dest is local interface address, but loopback device is not up\n");
dev_put(dev);
return -EINVAL;
}
/* Try default router */
if ((neigh = neigh_clone(dn_db->router)) != NULL)
......@@ -781,10 +796,12 @@ static int dn_route_output_slow(struct dst_entry **pprt, dn_address dst, dn_addr
if ((neigh = __neigh_lookup(&dn_neigh_table, &dst, dev, 1)) != NULL)
goto got_route;
dev_put(dev);
return -EINVAL;
got_route:
if (dev)
dev_put(dev);
if ((rt = dst_alloc(&dn_dst_ops)) == NULL) {
neigh_release(neigh);
......@@ -809,7 +826,7 @@ static int dn_route_output_slow(struct dst_entry **pprt, dn_address dst, dn_addr
rt->u.dst.output = dn_output;
rt->u.dst.input = dn_rt_bug;
if (dn_dev_islocal(neigh->dev, rt->rt_daddr))
if (neigh->dev->flags & IFF_LOOPBACK)
rt->u.dst.input = dn_nsp_rx;
hash = dn_hash(rt->key.saddr, rt->key.daddr);
......
......@@ -9,6 +9,7 @@
*
*
* Changes:
* Steve Whitehouse - C99 changes and default device handling
*
*/
#include <linux/config.h>
......@@ -152,7 +153,6 @@ static int dn_node_address_strategy(ctl_table *table, int *name, int nlen,
dn_dev_devices_off();
decnet_address = addr;
dn_dn2eth(decnet_ether_address, dn_ntohs(decnet_address));
dn_dev_devices_on();
}
......@@ -187,7 +187,6 @@ static int dn_node_address_handler(ctl_table *table, int write,
dn_dev_devices_off();
decnet_address = dnaddr;
dn_dn2eth(decnet_ether_address, dn_ntohs(decnet_address));
dn_dev_devices_on();
......@@ -218,9 +217,10 @@ static int dn_def_dev_strategy(ctl_table *table, int *name, int nlen,
void **context)
{
size_t len;
struct net_device *dev = decnet_default_device;
struct net_device *dev;
char devname[17];
size_t namel;
int rv = 0;
devname[0] = 0;
......@@ -228,8 +228,11 @@ static int dn_def_dev_strategy(ctl_table *table, int *name, int nlen,
if (get_user(len, oldlenp))
return -EFAULT;
if (len) {
if (dev)
dev = dn_dev_get_default();
if (dev) {
strcpy(devname, dev->name);
dev_put(dev);
}
namel = strlen(devname) + 1;
if (len > namel) len = namel;
......@@ -251,16 +254,19 @@ static int dn_def_dev_strategy(ctl_table *table, int *name, int nlen,
devname[newlen] = 0;
if ((dev = __dev_get_by_name(devname)) == NULL)
dev = dev_get_by_name(devname);
if (dev == NULL)
return -ENODEV;
if (dev->dn_ptr == NULL)
return -ENODEV;
decnet_default_device = dev;
rv = -ENODEV;
if (dev->dn_ptr != NULL) {
rv = dn_dev_set_default(dev, 1);
if (rv)
dev_put(dev);
}
}
return 0;
return rv;
}
......@@ -269,7 +275,7 @@ static int dn_def_dev_handler(ctl_table *table, int write,
void *buffer, size_t *lenp)
{
size_t len;
struct net_device *dev = decnet_default_device;
struct net_device *dev;
char devname[17];
if (!*lenp || (filp->f_pos && !write)) {
......@@ -287,24 +293,32 @@ static int dn_def_dev_handler(ctl_table *table, int write,
devname[*lenp] = 0;
strip_it(devname);
if ((dev = __dev_get_by_name(devname)) == NULL)
dev = dev_get_by_name(devname);
if (dev == NULL)
return -ENODEV;
if (dev->dn_ptr == NULL)
if (dev->dn_ptr == NULL) {
dev_put(dev);
return -ENODEV;
}
decnet_default_device = dev;
if (dn_dev_set_default(dev, 1)) {
dev_put(dev);
return -ENODEV;
}
filp->f_pos += *lenp;
return 0;
}
dev = dn_dev_get_default();
if (dev == NULL) {
*lenp = 0;
return 0;
}
strcpy(devname, dev->name);
dev_put(dev);
len = strlen(devname);
devname[len++] = '\n';
......@@ -320,51 +334,125 @@ static int dn_def_dev_handler(ctl_table *table, int write,
}
static ctl_table dn_table[] = {
{NET_DECNET_NODE_ADDRESS, "node_address", NULL, 7, 0644, NULL,
dn_node_address_handler, dn_node_address_strategy, NULL,
NULL, NULL},
{NET_DECNET_NODE_NAME, "node_name", node_name, 7, 0644, NULL,
&proc_dostring, &sysctl_string, NULL, NULL, NULL},
{NET_DECNET_DEFAULT_DEVICE, "default_device", NULL, 16, 0644, NULL,
dn_def_dev_handler, dn_def_dev_strategy, NULL, NULL, NULL},
{NET_DECNET_TIME_WAIT, "time_wait", &decnet_time_wait,
sizeof(int), 0644,
NULL, &proc_dointvec_minmax, &sysctl_intvec, NULL,
&min_decnet_time_wait, &max_decnet_time_wait},
{NET_DECNET_DN_COUNT, "dn_count", &decnet_dn_count,
sizeof(int), 0644,
NULL, &proc_dointvec_minmax, &sysctl_intvec, NULL,
&min_state_count, &max_state_count},
{NET_DECNET_DI_COUNT, "di_count", &decnet_di_count,
sizeof(int), 0644,
NULL, &proc_dointvec_minmax, &sysctl_intvec, NULL,
&min_state_count, &max_state_count},
{NET_DECNET_DR_COUNT, "dr_count", &decnet_dr_count,
sizeof(int), 0644,
NULL, &proc_dointvec_minmax, &sysctl_intvec, NULL,
&min_state_count, &max_state_count},
{NET_DECNET_DST_GC_INTERVAL, "dst_gc_interval", &decnet_dst_gc_interval,
sizeof(int), 0644,
NULL, &proc_dointvec_minmax, &sysctl_intvec, NULL,
&min_decnet_dst_gc_interval, &max_decnet_dst_gc_interval},
{NET_DECNET_NO_FC_MAX_CWND, "no_fc_max_cwnd", &decnet_no_fc_max_cwnd,
sizeof(int), 0644,
NULL, &proc_dointvec_minmax, &sysctl_intvec, NULL,
&min_decnet_no_fc_max_cwnd, &max_decnet_no_fc_max_cwnd},
{NET_DECNET_DEBUG_LEVEL, "debug", &decnet_debug_level,
sizeof(int), 0644,
NULL, &proc_dointvec, &sysctl_intvec, NULL,
NULL, NULL},
{
.ctl_name = NET_DECNET_NODE_ADDRESS,
.procname = "node_address",
.maxlen = 7,
.mode = 0644,
.proc_handler = dn_node_address_handler,
.strategy = dn_node_address_strategy,
},
{
.ctl_name = NET_DECNET_NODE_NAME,
.procname = "node_name",
.data = node_name,
.maxlen = 7,
.mode = 0644,
.proc_handler = &proc_dostring,
.strategy = &sysctl_string,
},
{
.ctl_name = NET_DECNET_DEFAULT_DEVICE,
.procname = "default_device",
.maxlen = 16,
.mode = 0644,
.proc_handler = dn_def_dev_handler,
.strategy = dn_def_dev_strategy,
},
{
.ctl_name = NET_DECNET_TIME_WAIT,
.procname = "time_wait",
.data = &decnet_time_wait,
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = &proc_dointvec_minmax,
.strategy = &sysctl_intvec,
.extra1 = &min_decnet_time_wait,
.extra2 = &max_decnet_time_wait
},
{
.ctl_name = NET_DECNET_DN_COUNT,
.procname = "dn_count",
.data = &decnet_dn_count,
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = &proc_dointvec_minmax,
.strategy = &sysctl_intvec,
.extra1 = &min_state_count,
.extra2 = &max_state_count
},
{
.ctl_name = NET_DECNET_DI_COUNT,
.procname = "di_count",
.data = &decnet_di_count,
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = &proc_dointvec_minmax,
.strategy = &sysctl_intvec,
.extra1 = &min_state_count,
.extra2 = &max_state_count
},
{
.ctl_name = NET_DECNET_DR_COUNT,
.procname = "dr_count",
.data = &decnet_dr_count,
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = &proc_dointvec_minmax,
.strategy = &sysctl_intvec,
.extra1 = &min_state_count,
.extra2 = &max_state_count
},
{
.ctl_name = NET_DECNET_DST_GC_INTERVAL,
.procname = "dst_gc_interval",
.data = &decnet_dst_gc_interval,
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = &proc_dointvec_minmax,
.strategy = &sysctl_intvec,
.extra1 = &min_decnet_dst_gc_interval,
.extra2 = &max_decnet_dst_gc_interval
},
{
.ctl_name = NET_DECNET_NO_FC_MAX_CWND,
.procname = "no_fc_max_cwnd",
.data = &decnet_no_fc_max_cwnd,
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = &proc_dointvec_minmax,
.strategy = &sysctl_intvec,
.extra1 = &min_decnet_no_fc_max_cwnd,
.extra2 = &max_decnet_no_fc_max_cwnd
},
{
.ctl_name = NET_DECNET_DEBUG_LEVEL,
.procname = "debug",
.data = &decnet_debug_level,
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = &proc_dointvec,
.strategy = &sysctl_intvec,
},
{0}
};
static ctl_table dn_dir_table[] = {
{NET_DECNET, "decnet", NULL, 0, 0555, dn_table},
{
.ctl_name = NET_DECNET,
.procname = "decnet",
.mode = 0555,
.child = dn_table},
{0}
};
static ctl_table dn_root_table[] = {
{CTL_NET, "net", NULL, 0, 0555, dn_dir_table},
{
.ctl_name = CTL_NET,
.procname = "net",
.mode = 0555,
.child = dn_dir_table
},
{0}
};
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment